Search Worker and I/O
The search worker is the orchestration layer for one search target. It owns the matcher, searcher, printer, command readers, and configuration needed to decide how input should reach the search pipeline.
It exists because searching a path is not always the same operation as searching a file directly: input may come from stdin, a preprocessor, a decompressor, or the path itself. The worker selects that route, applies binary-detection settings, and returns search results.
Sources: crates/core/search.rs:230-241, crates/core/search.rs:245-267
Core concepts
Search worker
A search worker coordinates search configuration, input preparation, matching, and output for one haystack. SearchWorker stores the PatternMatcher, Searcher, Printer, command-reader configuration, and optional decompression reader builder.
Sources: crates/core/search.rs:230-241
Search route
A search route is the worker’s choice among stdin, preprocessing, decompression, and direct path search. search checks those cases in that order after configuring binary detection.
Sources: crates/core/search.rs:245-267
Search result
A search result records whether a match occurred and, when available, printer statistics. SearchResult contains has_match and optional stats.
Sources: crates/core/search.rs:171-174
Command reader
A command reader tracks a child process, its stderr reader, and whether end-of-file has been reached. CommandReader stores those three pieces of state.
Sources: crates/cli/src/process.rs:171-178
How a search worker is assembled
SearchWorkerBuilder starts with default search configuration and a CommandReaderBuilder whose stderr handling is asynchronous. The builder can install a preprocessor, configure the globs that select it, enable archive decompression, and set separate implicit and explicit binary-detection policies.
The final build operation receives a PatternMatcher, Searcher, and Printer. If archive searching is enabled, it creates a DecompressionReaderBuilder and also enables asynchronous stderr handling for it; otherwise, the worker keeps no decompression builder.
| Configuration | Plain-language meaning |
|---|---|
preprocessor | Optional executable configuration for preprocessing. |
preprocessor_globs | Override rules deciding which paths use preprocessing. |
search_zip | Controls whether a decompression reader builder is created. |
binary_implicit | Binary-detection behavior for targets found implicitly. |
binary_explicit | Binary-detection behavior for explicitly requested targets. |
These fields are the worker’s routing and binary-policy inputs.
The worker binds the matcher, searcher, and printer into one operation: PatternMatcher represents the regex backend, while Printer selects standard, summary, or JSON output.
Sources: crates/core/search.rs:54-59, crates/core/search.rs:92-103, crates/core/search.rs:108-114, crates/core/search.rs:123-129, crates/core/search.rs:139-145, crates/core/search.rs:155-161, crates/core/search.rs:63-85, crates/core/search.rs:193-197, crates/core/search.rs:203-211
How input is routed
The following sequence shows the worker’s hand-off from route selection into reader-based searching and printer-sink creation.
Evidence
- search-workercrates/core/search.rs:245
- searchercrates/core/search.rs:416
- printer-sinkcrates/core/search.rs:423
The route decision uses the target’s explicitness to select binary detection first. Stdin is searched directly; otherwise, a matching preprocessor takes priority, then a matching decompressor, and finally the path is searched directly.
| Route | Condition | Worker call |
|---|---|---|
| Stdin | is_stdin() is true | search_reader |
| Preprocessor | should_preprocess(path) is true | search_preprocessor |
| Decompressor | should_decompress(path) is true | search_decompress |
| Direct path | None of the earlier conditions apply | search_path |
The route order is significant: preprocessing wins over decompression when both could otherwise apply.
This data-flow view shows the worker’s four visible route hand-offs before each path reaches its corresponding search implementation.
Evidence
- workercrates/core/search.rs:245
- stdin-routecrates/core/search.rs:245
- preprocess-routecrates/core/search.rs:296
- decompress-routecrates/core/search.rs:329
- path-routecrates/core/search.rs:342
should_preprocess returns false without a configured preprocessor. With no preprocessor globs, it applies to every path; with globs, it applies when the path is not ignored by the override matcher. should_decompress is true only when the optional decompression builder exists and its matcher has a command for the path.
Sources: crates/core/search.rs:245-267, crates/core/search.rs:416-426, crates/core/search.rs:296-324, crates/core/search.rs:329-339, crates/core/search.rs:342-351, crates/core/search.rs:284-292, crates/core/search.rs:276-280
How preprocessing and decompression read data
The preprocessor route constructs a command with the configured executable, passes the path as an argument, and supplies the original file through the command’s stdin. It then passes the resulting reader to search_reader, closes the reader, and returns the search result only after both search and close succeed.
Decompression follows the same reader-oriented shape, but selects a command from a filename glob. DecompressionMatcher stores globs paired with commands, and command returns the command associated with the last matching glob. DecompressionReaderBuilder::build appends the path to that command and starts it through the command builder.
If no decompression command matches, the builder creates a pass-through reader over the original file. If spawning the decompressor fails, it logs a debug message and also falls back to the uncompressed reader.
The built-in decompression associations cover gzip, bzip2, xz, lz4, lzma, brotli, zstd, and Unix-compress filename patterns, subject to resolving the corresponding executable. Binary resolution returns an absolute executable directly or searches the system PATH; on failure it returns an I/O-shaped CommandError.
Both preprocessors and decompressors use child-process readers. CommandReaderBuilder::build pipes stdout and stderr, spawns the child, and chooses asynchronous or synchronous stderr handling. When stdout reaches EOF, CommandReader::read closes the child and returns zero bytes. Closing drops stdout, waits for the child, and examines stderr when the child does not exit successfully.
Sources: crates/core/search.rs:296-324, crates/cli/src/decompress.rs:147-154, crates/cli/src/decompress.rs:179-187, crates/cli/src/decompress.rs:221-244, crates/cli/src/decompress.rs:361-364, crates/cli/src/decompress.rs:490-532, crates/cli/src/decompress.rs:448-488, crates/cli/src/process.rs:104-118, crates/cli/src/process.rs:255-267, crates/cli/src/process.rs:218-243
How results and warnings reach the user
Once a reader is selected, the worker delegates to search_reader. That method chooses the matcher variant and passes the matcher, searcher, printer, path, and reader to the generic search function. The generic function creates a path-aware sink, invokes searcher.search_reader, and extracts match and statistics from that sink.
Direct path searches use the equivalent sink flow through searcher.search_path. Standard and summary printers return optional cloned statistics, while the JSON printer always returns cloned statistics.
Binary detection is selected separately for explicit and implicit targets before routing begins. The searcher receives that choice through set_binary_detection. The documented behavior is that a binary search may stop early and warn when matches are suppressed or when searching stops prematurely after a match.
The shown message path is gated by the crate::messages::messages() function: message calls eprintln_locked only when that function returns true. Ignore-related output has an additional crate::messages::ignore_messages() check in ignore_message. Error messages use err_message, which marks the global errored state before emitting the message.
The locked stderr path prefixes output with rg:, handles broken pipes by exiting gracefully, and avoids interleaving with search output. The excerpts establish the generic message gate and the binary-warning behavior separately; they do not show the binary-warning call site.
Sources: crates/core/search.rs:362-375, crates/core/search.rs:416-449, crates/core/search.rs:380-412, crates/core/search.rs:245-267, crates/core/flags/defs.rs:524-540, crates/core/messages.rs:71-77, crates/core/messages.rs:92-98, crates/core/messages.rs:82-87, crates/core/messages.rs:33-67
How it connects
Directory traversal produces candidate haystacks and dispatches them to workers; explicit entries are always retained, while ordinary entries must pass the file filter. See Parallel Directory Traversal.
The worker supplies the Searcher with a PatternMatcher, while the matcher implementations provide the low-level pattern operations. See The Matcher Trait and Internal Iteration, The Default Rust Regex Matcher, and The PCRE2 Matcher.
The Searcher drives sinks, and the worker’s Printer supplies the concrete output sink. See The Searcher Core and Output Printers.
The command-line layer resolves preprocessing, archive-search, and binary-detection settings before constructing the worker. See CLI Entry and Flag Parsing.
Sources: crates/core/haystack.rs:49-79
Key takeaways
SearchWorkercoordinates configuration, input preparation, matching, and printing.- Route selection is stdin, preprocessor, decompressor, then direct path search.
- Preprocessors and decompressors feed the same reader-based search path.
- Decompression falls back to the original file when no command matches or spawning fails.
- Binary-search behavior includes warnings for suppressed matches or premature stopping, while the shown generic message path is gated by
messages().
Sources: crates/core/search.rs:230-241, crates/core/search.rs:245-267, crates/core/search.rs:296-324, crates/core/search.rs:329-339, crates/cli/src/decompress.rs:221-244, crates/cli/src/decompress.rs:361-364, crates/core/flags/defs.rs:524-540, crates/core/messages.rs:71-77