3 [![Build Status](https://travis-ci.org/andrewrk/node-fd-slicer.svg?branch=master)](https://travis-ci.org/andrewrk/node-fd-slicer)
4 [![Coverage Status](https://img.shields.io/coveralls/andrewrk/node-fd-slicer.svg)](https://coveralls.io/r/andrewrk/node-fd-slicer)
6 Safe `fs.ReadStream` and `fs.WriteStream` using the same fd.
8 Let's say that you want to perform a parallel upload of a file to a remote
9 server. To do this, we want to create multiple read streams. The first thing
10 you might think of is to use the `{start: 0, end: 0}` API of
11 `fs.createReadStream`. This gives you two choices:
13 0. Use the same file descriptor for all `fs.ReadStream` objects.
14 0. Open the file multiple times, resulting in a separate file descriptor
17 Neither of these are acceptable options. The first one is a severe bug,
18 because the API docs for `fs.write` state:
20 > Note that it is unsafe to use `fs.write` multiple times on the same file
21 > without waiting for the callback. For this scenario, `fs.createWriteStream`
22 > is strongly recommended.
24 `fs.createWriteStream` will solve the problem if you only create one of them
25 for the file descriptor, but it will exhibit this unsafety if you create
26 multiple write streams per file descriptor.
28 The second option suffers from a race condition. For each additional time the
29 file is opened after the first, it is possible that the file is modified. So
30 in our parallel uploading example, we might upload a corrupt file that never
31 existed on the client's computer.
33 This module solves this problem by providing `createReadStream` and
34 `createWriteStream` that operate on a shared file descriptor and provides
35 the convenient stream API while still allowing slicing and dicing.
37 This module also gives you some additional power that the builtin
38 `fs.createWriteStream` do not give you. These features are:
40 * Emitting a 'progress' event on write.
41 * Ability to set a maximum size and emit an error if this size is exceeded.
42 * Ability to create an `FdSlicer` instance from a `Buffer`. This enables you
43 to provide API for handling files as well as buffers using the same API.
48 var fdSlicer = require('fd-slicer');
49 var fs = require('fs');
51 fs.open("file.txt", 'r', function(err, fd) {
53 var slicer = fdSlicer.createFromFd(fd);
54 var firstPart = slicer.createReadStream({start: 0, end: 100});
55 var secondPart = slicer.createReadStream({start: 100});
56 var firstOut = fs.createWriteStream("first.txt");
57 var secondOut = fs.createWriteStream("second.txt");
58 firstPart.pipe(firstOut);
59 secondPart.pipe(secondOut);
63 You can also create from a buffer:
66 var fdSlicer = require('fd-slicer');
67 var slicer = FdSlicer.createFromBuffer(someBuffer);
68 var firstPart = slicer.createReadStream({start: 0, end: 100});
69 var secondPart = slicer.createReadStream({start: 100});
70 var firstOut = fs.createWriteStream("first.txt");
71 var secondOut = fs.createWriteStream("second.txt");
72 firstPart.pipe(firstOut);
73 secondPart.pipe(secondOut);
78 ### fdSlicer.createFromFd(fd, [options])
81 var fdSlicer = require('fd-slicer');
82 fs.open("file.txt", 'r', function(err, fd) {
84 var slicer = fdSlicer.createFromFd(fd);
89 Make sure `fd` is a properly initialized file descriptor. If you want to
90 use `createReadStream` make sure you open it for reading and if you want
91 to use `createWriteStream` make sure you open it for writing.
93 `options` is an optional object which can contain:
95 * `autoClose` - if set to `true`, the file descriptor will be automatically
96 closed once the last stream that references it is closed. Defaults to
97 `false`. `ref()` and `unref()` can be used to increase or decrease the
98 reference count, respectively.
100 ### fdSlicer.createFromBuffer(buffer)
103 var fdSlicer = require('fd-slicer');
104 var slicer = fdSlicer.createFromBuffer(someBuffer);
112 The file descriptor passed in. `undefined` if created from a buffer.
116 ##### createReadStream(options)
120 * `start` - Number. The offset into the file to start reading from. Defaults
122 * `end` - Number. Exclusive upper bound offset into the file to stop reading
124 * `highWaterMark` - Number. The maximum number of bytes to store in the
125 internal buffer before ceasing to read from the underlying resource.
127 * `encoding` - String. If specified, then buffers will be decoded to strings
128 using the specified encoding. Defaults to `null`.
130 The ReadableStream that this returns has these additional methods:
132 * `destroy(err)` - stop streaming. `err` is optional and is the error that
133 will be emitted in order to cause the streaming to stop. Defaults to
134 `new Error("stream destroyed")`.
136 ##### createWriteStream(options)
140 * `start` - Number. The offset into the file to start writing to. Defaults to
142 * `end` - Number. Exclusive upper bound offset into the file. If this offset
143 is reached, the write stream will emit an 'error' event and stop functioning.
144 In this situation, `err.code === 'ETOOBIG'`. Defaults to `Infinity`.
145 * `highWaterMark` - Number. Buffer level when `write()` starts returning
146 false. Defaults to 16KB.
147 * `decodeStrings` - Boolean. Whether or not to decode strings into Buffers
148 before passing them to` _write()`. Defaults to `true`.
150 The WritableStream that this returns has these additional methods:
152 * `destroy()` - stop streaming
154 And these additional properties:
156 * `bytesWritten` - number of bytes written to the stream
158 And these additional events:
160 * 'progress' - emitted when `bytesWritten` changes.
162 ##### read(buffer, offset, length, position, callback)
164 Equivalent to `fs.read`, but with concurrency protection.
165 `callback` must be defined.
167 ##### write(buffer, offset, length, position, callback)
169 Equivalent to `fs.write`, but with concurrency protection.
170 `callback` must be defined.
174 Increase the `autoClose` reference count by 1.
178 Decrease the `autoClose` reference count by 1.
184 Emitted if `fs.close` returns an error when auto closing.
188 Emitted when fd-slicer closes the file descriptor due to `autoClose`. Never
189 emitted if created from a buffer.