You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
210 lines
16 KiB
210 lines
16 KiB
11 months ago
|
<!DOCTYPE html>
|
||
|
|
<html lang="en">
|
||
|
|
<head>
|
||
|
|
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||
|
|
<meta http-equiv="content-type" content="text/html; charset=utf-8">
|
||
|
|
|
||
|
|
<!-- Enable responsiveness on mobile devices-->
|
||
|
|
<!-- viewport-fit=cover is to support iPhone X rounded corners and notch in landscape-->
|
||
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1, viewport-fit=cover">
|
||
|
|
|
||
|
|
<title>Julio Biason .Me 4.3</title>
|
||
|
|
|
||
|
|
<!-- CSS -->
|
||
|
|
<link rel="stylesheet" href="https://blog.juliobiason.me/print.css" media="print">
|
||
|
|
<link rel="stylesheet" href="https://blog.juliobiason.me/poole.css">
|
||
|
|
<link rel="stylesheet" href="https://blog.juliobiason.me/hyde.css">
|
||
|
|
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=PT+Sans:400,400italic,700|Abril+Fatface">
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
</head>
|
||
|
|
|
||
|
|
<body class=" ">
|
||
|
|
|
||
|
|
<div class="sidebar">
|
||
|
|
<div class="container sidebar-sticky">
|
||
|
|
<div class="sidebar-about">
|
||
|
|
|
||
|
|
<a href="https://blog.juliobiason.me"><h1>Julio Biason .Me 4.3</h1></a>
|
||
|
|
|
||
|
|
<p class="lead">Old school dev living in a 2.0 dev world</p>
|
||
|
|
|
||
|
|
|
||
|
|
</div>
|
||
|
|
|
||
|
|
<ul class="sidebar-nav">
|
||
|
|
|
||
|
|
|
||
|
|
<li class="sidebar-nav-item"><a href="/">English</a></li>
|
||
|
|
|
||
|
|
<li class="sidebar-nav-item"><a href="/pt">Português</a></li>
|
||
|
|
|
||
|
|
<li class="sidebar-nav-item"><a href="/tags">Tags (EN)</a></li>
|
||
|
|
|
||
|
|
<li class="sidebar-nav-item"><a href="/pt/tags">Tags (PT)</a></li>
|
||
|
|
|
||
|
|
|
||
|
|
</ul>
|
||
|
|
</div>
|
||
|
|
</div>
|
||
|
|
|
||
|
|
|
||
|
|
<div class="content container">
|
||
|
|
|
||
|
|
<div class="post">
|
||
|
|
<h1 class="post-title">Running a Command and Saving Its Output to File in Rust</h1>
|
||
|
|
<span class="post-date">
|
||
|
|
2023-09-01
|
||
|
|
|
||
|
|
<a href="https://blog.juliobiason.me/tags/random/">#random</a>
|
||
|
|
|
||
|
|
<a href="https://blog.juliobiason.me/tags/rust/">#rust</a>
|
||
|
|
|
||
|
|
<a href="https://blog.juliobiason.me/tags/command/">#command</a>
|
||
|
|
|
||
|
|
<a href="https://blog.juliobiason.me/tags/log/">#log</a>
|
||
|
|
|
||
|
|
</span>
|
||
|
|
<p>I had an issue: I needed to run a command inside Rust, but I needed that all
|
||
|
|
its output should go to a file, and I needed to check if there were certain
|
||
|
|
phrases in it.</p>
|
||
|
|
<span id="continue-reading"></span>
|
||
|
|
<p>So, first step: Create a script that could "replicate" the output of a command,
|
||
|
|
with the expected strings to be captured:</p>
|
||
|
|
<pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#65737e;">#!/usr/bin/env bash
|
||
|
|
</span><span>
|
||
|
|
</span><span style="color:#b48ead;">for</span><span> loop </span><span style="color:#b48ead;">in </span><span>{1..1000}
|
||
|
|
</span><span style="color:#b48ead;">do
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;">Hello, I'm a script!</span><span>"
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;">I write stuff in the output.</span><span>"
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;">Everything should go to a file.</span><span>"
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;">But also, you need to capture warnings:</span><span>"
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#b48ead;">if </span><span>(( $</span><span style="color:#bf616a;">loop</span><span>%</span><span style="color:#d08770;">7 </span><span>== </span><span style="color:#d08770;">0</span><span>)); </span><span style="color:#b48ead;">then
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;">WARNING: This is a warning</span><span>"
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;"> It continues if the line starts with spaces</span><span>"
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;"> And keeps going till there are no more spaces-prefixes</span><span>"
|
||
|
|
</span><span> </span><span style="color:#b48ead;">fi
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#b48ead;">if </span><span>(( $</span><span style="color:#bf616a;">loop</span><span>%</span><span style="color:#d08770;">8 </span><span>== </span><span style="color:#d08770;">0</span><span>)); </span><span style="color:#b48ead;">then
|
||
|
|
</span><span> </span><span style="color:#65737e;"># ERR is just to make sure we find it easily in the logs
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;">ERR: Sometimes, I also write in stderr!</span><span>" >&</span><span style="color:#d08770;">2
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;">ERR: Just for funsies!</span><span>" >&</span><span style="color:#d08770;">2
|
||
|
|
</span><span> </span><span style="color:#b48ead;">fi
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;">Like this.</span><span>"
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>"</span><span style="color:#a3be8c;">Then you're good to go.</span><span>"
|
||
|
|
</span><span> </span><span style="color:#96b5b4;">echo </span><span>""
|
||
|
|
</span><span style="color:#b48ead;">done
|
||
|
|
</span></code></pre>
|
||
|
|
<p>What this script does is to print a message over 1,000 times, and sometimes it
|
||
|
|
will display a "WARNING" text -- which is the special output I need to capture --
|
||
|
|
and sometimes it will print things to stderr.</p>
|
||
|
|
<p>For the code, what we need to do is:</p>
|
||
|
|
<ol>
|
||
|
|
<li>Spawn the command;</li>
|
||
|
|
<li>Take the stdour (and stderr) from it.</li>
|
||
|
|
<li>Spawn a thread that will keep listening to the output, doing the search,
|
||
|
|
and writing everything to a file (our log).</li>
|
||
|
|
<li>The thread returns the list of captured messages, which we can get back
|
||
|
|
when we <code>.join()</code> it.</li>
|
||
|
|
<li>Since I was expecting stderr to be smaller enough, I did the capturing of
|
||
|
|
it after the thread completes (which would also close the file, so we can
|
||
|
|
be sure that we can open it again without any issues).</li>
|
||
|
|
</ol>
|
||
|
|
<p>The first step is quite easy: Just use <code>std::process::Command</code> and use the
|
||
|
|
<code>.spawn()</code> function to create the <code>Child</code> controller.</p>
|
||
|
|
<p>For the second step, we use the <code>Child</code> structure and use <code>.take()</code> on both
|
||
|
|
stdout and stderr. This will give us the file descriptor for both (think about
|
||
|
|
them as <code>File</code>s).</p>
|
||
|
|
<p>The third step is quite easy, actualy: <code>std::thread::spawn()</code> to create a
|
||
|
|
thread, and just read the content from the file descriptors from step 2. In
|
||
|
|
this, I used <code>BufReader</code>, which gives access to reading the content line by
|
||
|
|
line, which is way easier than reading to a buffer and processing it.</p>
|
||
|
|
<pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>std::fs::{File, OpenOptions};
|
||
|
|
</span><span style="color:#b48ead;">use </span><span>std::io::{BufRead, BufReader, Read, Write};
|
||
|
|
</span><span style="color:#b48ead;">use </span><span>std::path::PathBuf;
|
||
|
|
</span><span style="color:#b48ead;">use </span><span>std::process::Command;
|
||
|
|
</span><span>
|
||
|
|
</span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() {
|
||
|
|
</span><span> </span><span style="color:#65737e;">// this requires always running with `cargo run`
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let</span><span> base = PathBuf::from(env!("</span><span style="color:#a3be8c;">CARGO_MANIFEST_DIR</span><span>"));
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let</span><span> the_script = base.</span><span style="color:#96b5b4;">join</span><span>("</span><span style="color:#a3be8c;">src</span><span>").</span><span style="color:#96b5b4;">join</span><span>("</span><span style="color:#a3be8c;">the_script.sh</span><span>");
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let mut</span><span> cmd = Command::new("</span><span style="color:#a3be8c;">bash</span><span>")
|
||
|
|
</span><span> .</span><span style="color:#96b5b4;">arg</span><span>(the_script)
|
||
|
|
</span><span> .</span><span style="color:#96b5b4;">stdout</span><span>(std::process::Stdio::piped())
|
||
|
|
</span><span> .</span><span style="color:#96b5b4;">stderr</span><span>(std::process::Stdio::piped())
|
||
|
|
</span><span> .</span><span style="color:#96b5b4;">spawn</span><span>()
|
||
|
|
</span><span> .</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#65737e;">// capture both the stdout and stderr as File structs (actually FDs, but basically the same
|
||
|
|
</span><span> </span><span style="color:#65737e;">// thing)
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let</span><span> stdout = cmd.stdout.</span><span style="color:#96b5b4;">take</span><span>().</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let mut</span><span> stderr = cmd.stderr.</span><span style="color:#96b5b4;">take</span><span>().</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#65737e;">// spawn a thread to keep capturing and processing the stdout.
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let</span><span> writer_pid = std::thread::spawn(</span><span style="color:#b48ead;">move </span><span>|| {
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let</span><span> reader = BufReader::new(stdout);
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let</span><span> lines = reader.</span><span style="color:#96b5b4;">lines</span><span>();
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let mut</span><span> log_file = File::create("</span><span style="color:#a3be8c;">script.log</span><span>").</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let mut</span><span> in_warning = </span><span style="color:#d08770;">false</span><span>;
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let mut</span><span> result = Vec::new();
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#b48ead;">for</span><span> line in lines {
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let</span><span> line = line.</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span> log_file.</span><span style="color:#96b5b4;">write</span><span>(line.</span><span style="color:#96b5b4;">as_bytes</span><span>()).</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span> log_file.</span><span style="color:#96b5b4;">write</span><span>(</span><span style="color:#b48ead;">b</span><span>"</span><span style="color:#96b5b4;">\n</span><span>").</span><span style="color:#96b5b4;">unwrap</span><span>(); </span><span style="color:#65737e;">// 'cause lines() eat it
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#b48ead;">if</span><span> line.</span><span style="color:#96b5b4;">starts_with</span><span>("</span><span style="color:#a3be8c;">WARNING:</span><span>") {
|
||
|
|
</span><span> in_warning = </span><span style="color:#d08770;">true</span><span>;
|
||
|
|
</span><span> } </span><span style="color:#b48ead;">else if</span><span> line.</span><span style="color:#96b5b4;">starts_with</span><span>(" ") && in_warning {
|
||
|
|
</span><span> result.</span><span style="color:#96b5b4;">push</span><span>(line);
|
||
|
|
</span><span> } </span><span style="color:#b48ead;">else if</span><span> in_warning {
|
||
|
|
</span><span> in_warning = </span><span style="color:#d08770;">false</span><span>;
|
||
|
|
</span><span> }
|
||
|
|
</span><span> }
|
||
|
|
</span><span>
|
||
|
|
</span><span> result
|
||
|
|
</span><span> });
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#65737e;">// run the command till it finishes
|
||
|
|
</span><span> cmd.</span><span style="color:#96b5b4;">wait</span><span>().</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#65737e;">// ... and wait till the thread finishes processing the whole output.
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let</span><span> warnings = writer_pid.</span><span style="color:#96b5b4;">join</span><span>().</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#65737e;">// this is somewhat a hack: Instead of spawning a thread for stderr and trying to fight with
|
||
|
|
</span><span> </span><span style="color:#65737e;">// stdout for the lock to be able to write in the log file, we do this after the thread ends
|
||
|
|
</span><span> </span><span style="color:#65737e;">// (which closes the file) and then open it again and write the stderr in the end. We do this
|
||
|
|
</span><span> </span><span style="color:#65737e;">// 'cause we expect that the stderr is way smaller than stdout and can fit in memory without
|
||
|
|
</span><span> </span><span style="color:#65737e;">// any issues.
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let mut</span><span> buffer = String::new();
|
||
|
|
</span><span> stderr.</span><span style="color:#96b5b4;">read_to_string</span><span>(&</span><span style="color:#b48ead;">mut</span><span> buffer).</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#b48ead;">let mut</span><span> file = OpenOptions::new().</span><span style="color:#96b5b4;">append</span><span>(</span><span style="color:#d08770;">true</span><span>).</span><span style="color:#96b5b4;">open</span><span>("</span><span style="color:#a3be8c;">script.log</span><span>").</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span> file.</span><span style="color:#96b5b4;">write</span><span>(buffer.</span><span style="color:#96b5b4;">as_bytes</span><span>()).</span><span style="color:#96b5b4;">unwrap</span><span>();
|
||
|
|
</span><span>
|
||
|
|
</span><span> </span><span style="color:#65737e;">// This is purely for diagnostic purposes. We could put the warnings in another file, or pass
|
||
|
|
</span><span> </span><span style="color:#65737e;">// it along to something else to process it. Here, we just display them.
|
||
|
|
</span><span> </span><span style="color:#65737e;">// Same for stderr: Since we already put them in the file, this is used just to make sure we
|
||
|
|
</span><span> </span><span style="color:#65737e;">// are capturing the errors without looking at the file.
|
||
|
|
</span><span> println!("</span><span style="color:#a3be8c;">Warnings:</span><span style="color:#96b5b4;">\n</span><span style="color:#d08770;">{:?}</span><span>", warnings);
|
||
|
|
</span><span> println!("</span><span style="color:#a3be8c;">ERR:</span><span style="color:#96b5b4;">\n</span><span style="color:#d08770;">{:?}</span><span>", buffer)
|
||
|
|
</span><span>}
|
||
|
|
</span></code></pre>
|
||
|
|
|
||
|
|
</div>
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
</div>
|
||
|
|
|
||
|
|
</body>
|
||
|
|
|
||
|
|
</html>
|