C# File I/O — Missing `using` Locked Production API
Production API failed with 'file in use' IOException from a missing using block.
- C# File I/O offers three layers: File static class, StreamReader/StreamWriter, and FileStream
- File.ReadAllText loads everything into memory – use only for files under ~10MB
- StreamReader.ReadLine() keeps memory flat regardless of file size, essential for unbounded files
- Async variants (ReadAllTextAsync, ReadLineAsync) release threads during disk wait, preventing thread-pool starvation under load
- Not disposing streams leaves files locked – leads to IOException in production bug reports
- Biggest mistake: using File.ReadAllLines on user-uploaded CSVs – server crashes with OutOfMemoryException
Think of your hard drive as a giant filing cabinet. Your C# program is the office worker who needs to pull out a document, read it, maybe scribble some notes on it, and then put it back. File I/O is simply the set of instructions that tells that office worker HOW to open the drawer, handle the document carefully, and close the drawer when done — without losing any pages or jamming the cabinet.
Every meaningful application eventually needs to talk to the file system. Whether you're building a log aggregator, a config-file reader, a report exporter, or a data pipeline that processes CSV files overnight — the moment your app needs to persist something beyond memory, File I/O is what stands between you and a working product. Yet it's one of those topics where developers confidently write code that works on their machine and silently fails in production because of an unclosed stream, a missing directory, or a race condition with another process.
The .NET runtime gives you a surprisingly rich toolbox for file operations, and the problem isn't a lack of options — it's knowing which tool is right for which job. Should you use File.ReadAllText or StreamReader? Should your read operation be synchronous or async? What happens when the file doesn't exist yet, or when two threads try to write to it at the same time? These are the questions that separate code that ships from code that apologizes.
By the end of this article you'll understand the full lifecycle of a file operation in C#, know exactly when to reach for each API in the toolbox, write async file code that doesn't deadlock, and handle the most common real-world edge cases with confidence. The code examples here are production-grade patterns, not toy demos.
The Three Layers of File I/O in C# — and Why They Exist
C# gives you three distinct levels of abstraction for file work, each built on top of the one below it. Understanding this layering is what stops you from grabbing the wrong tool.
At the lowest level you have FileStream — raw bytes, maximum control, maximum verbosity. Above that sit StreamReader and StreamWriter, which wrap a FileStream and add character encoding and line-by-line text handling. At the top sits the static File class, which wraps everything into single-line convenience methods like File.ReadAllText and File.WriteAllLines.
The File class is perfect for small files where simplicity matters — it opens the file, does the work, and closes it all in one call. But it reads the entire file into memory at once, which is a problem when that file is 2 GB of server logs. That's when you drop down to StreamReader and read line by line, keeping your memory footprint flat regardless of file size.
FileStream is the layer you reach for when you need binary data — images, PDFs, serialized objects — or when you need fine-grained control over file sharing modes and access permissions.
Most real-world apps live in the middle layer. Know that the File convenience methods are literally just wrappers around streams — there's no magic, just convenience.
File.ReadAllText / File.WriteAllText for files under ~10 MB where simplicity wins. Switch to StreamReader / StreamWriter the moment file size is unbounded or user-controlled — an uploaded CSV could be 500 MB.StreamReader.ReadLine(), maintaining constant memory usage with no code complexity cost.Async File I/O — Why Blocking a Thread on Disk Reads is a Hidden Performance Killer
Here's the thing most tutorials skip: disk I/O is slow. Not 'slightly slower than memory' slow — we're talking microseconds vs milliseconds. On a web server handling 500 concurrent requests, if each request reads a file synchronously, each one blocks a thread for that entire disk-wait time. Thread pool threads are a finite resource. Block enough of them and your server stops accepting new requests even though the CPU is sitting at 2% utilisation.
Async file I/O solves this by releasing the thread back to the pool while it waits for the disk. The thread goes off and serves other requests. When the disk responds, .NET picks up any available thread to continue the work.
File.ReadAllTextAsync and StreamReader.ReadLineAsync are the async counterparts you need. They return Task<string> and Task<string?> respectively, meaning you await them without blocking.
One critical nuance: StreamReader does NOT automatically buffer async reads efficiently when you call ReadLineAsync repeatedly in a tight loop on .NET 5 and earlier. On .NET 6+ this was fixed. If you're on an older runtime, prefer ReadToEndAsync or use FileStream with useAsync: true directly.
Async file operations belong in any application that handles concurrent workloads — ASP.NET Core controllers, background workers, and queue processors absolutely should not use synchronous file APIs.
async void instead of async Task for file methods means any exception thrown during the async operation is unobservable — it won't be caught by your try/catch and will silently crash the process. Always return Task or Task<T> from async file methods.Defensive File I/O — Handling Missing Files, Locked Resources and Directory Errors
Production file code fails in ways your dev machine never shows you. The config file doesn't exist on first run. The log directory hasn't been created yet. Another process has locked the file. The disk is full. A relative path resolves to a completely different location when deployed.
Defensive file I/O means anticipating these realities before they become 3am incident alerts.
The key exceptions to know are FileNotFoundException (file doesn't exist), DirectoryNotFoundException (parent directory missing), IOException (file locked, disk full, network drive disconnected), and UnauthorizedAccessException (permissions). Catching the base IOException catches most of them, but be specific when the recovery action differs.
For directories: always call Directory.CreateDirectory before writing — it's idempotent and won't throw if the directory already exists. This one pattern eliminates an entire class of deployment bugs.
For locked files: the right pattern is a retry loop with exponential back-off, not a bare try/catch that swallows the error. A locked file often means another process is actively writing to it and will be done in milliseconds.
For paths: use Path.Combine instead of string concatenation — it handles directory separators correctly across Windows, Linux, and macOS. Hardcoded backslashes are a cross-platform bug waiting to happen.
Directory.CreateDirectory is idempotent — calling it when the directory already exists doesn't throw an exception. This makes it safe as a defensive first step before any file write, no Directory.Exists check required.Working with CSV and Structured Text Files — A Real-World End-to-End Pattern
Almost every business application eventually processes CSV files — imports, exports, data migrations. This is where all the concepts above converge into a pattern you'll actually use.
The key insight for large CSV processing is streaming: read one line at a time, process it, move on. Never ReadAllLines a CSV that users upload — you're handing users a memory exhaustion attack vector. A 100 MB CSV with ReadAllLines allocates all 100 MB at once. With StreamReader.ReadLine you hold one line in memory at a time.
Encoding also matters in the real world. CSVs from Windows systems often arrive in Windows-1252 encoding. CSVs from Excel often have a UTF-8 BOM. StreamReader can auto-detect the BOM if you pass detectEncodingFromByteOrderMarks: true, which saves you from mysterious £ characters replacing £ signs.
For writing, StreamWriter with AutoFlush = false is dramatically faster than flushing after every line — let the OS buffer accumulate and flush at natural boundaries. If the process dies mid-write you'll lose the buffer, so pair this with a write-to-temp-file-then-rename pattern for atomicity.
The temp-file-then-rename pattern is the professional's choice for any file that must not be corrupted if the process dies mid-write: write to report.tmp, then File.Move("report.tmp", "report.csv", overwrite: true). The OS rename is atomic on most filesystems.
File.Move with overwrite: true (available from .NET 3.0) makes it a one-liner. Use it for any file that another system depends on.File Locking and Concurrent Access — Protecting Shared Resources
When multiple processes or threads try to access the same file, you need to think about concurrency. The default FileShare mode is FileShare.Read, which allows other processes to read the file while your stream is open for writing. But if two threads write to the same file simultaneously, you'll get data corruption or exceptions.
For a single process, use the lock statement to ensure only one thread writes at a time. For cross-process coordination, you'll need a named Mutex or a dedicated file-locking mechanism.
The FileStream constructor accepts a FileShare parameter that controls what other processes can do while your handle is open. Common combinations: - FileMode.Open, FileAccess.Read, FileShare.Read – multiple readers, no writers. - FileMode.OpenOrCreate, FileAccess.Write, FileShare.Read – exclusive write, others can read. - FileMode.Open, FileAccess.ReadWrite, FileShare.None – exclusive access.
For high-concurrency logging, use a dedicated logging library (Serilog, NLog) that handles file locking internally. Writing your own lock-based file access is a recipe for deadlocks and performance issues.
If you must write to a shared file, use File.AppendAllText or File.AppendAllTextAsync – they open, append, and close in one atomic operation, minimising the window for contention.
lock won't work because it's per-process. Use a named Mutex or rely on a file-locking mechanism like FileStream with FileShare.None. But ideally, use a logging library or a message queue instead of sharing files.lock statement around file writes. Avoid holding the lock for long operations.File.AppendAllText – it opens, appends, and closes atomically, reducing contention window.The Locked Log File: How a Missing `using` Statement Brought Down a Production API
using block. The garbage collector eventually finalizes the object, but not before the file handle remains open for an indeterminate time, causing contention with health checks.using statements (or await using), ensuring immediate release of the file handle.- Always wrap IDisposable file objects in using blocks.
- Never rely on garbage collection to close file handles — it's non-deterministic.
- Use File.AppendAllText for simple appends to avoid manual stream management.
detectEncodingFromByteOrderMarks: true to handle UTF-8 BOM. Avoid platform default encoding.using blocks immediately. If using async, use await using.Key takeaways
Common mistakes to avoid
3 patternsNot disposing StreamReader/StreamWriter
Dispose() even if an exception is thrown, which flushes the buffer and releases the OS file handle.Using File.ReadAllLines on user-uploaded or unbounded files
StreamReader.ReadLine() in a while loop. You hold one line in memory at a time. If you need IEnumerable<string> semantics, wrap it in a generator method with yield return.Building file paths with string concatenation
Interview Questions on This Topic
What's the difference between File.ReadAllText and StreamReader, and when would you choose one over the other in a production application?
Frequently Asked Questions
That's C# Basics. Mark it forged?
6 min read · try the examples if you haven't