General documentation / cheat sheets for various languages and services

Node.js: stream

There are a number of conventions associated with streams, which have pretty minimal requirements. Suppose you have a var stream = fs.createReadStream('README.md'); in this case, stream.autoClose = true by default. Until you call stream.read(1000), stream.readable = true, and stream.destroyed = stream.closed = undefined.

Once you read to the end, i.e., stream.read(size) returns a Buffer shorter than size, or null (I kinda wish it would return a 0-length Buffer, but no; not even if you open the stream with {autoClose: false}), stream.readable = false, and stream.destroyed = stream.closed = true.

The fs.ReadStream instance returned by fs.createReadStream() also has three helper functions on top of its stream.Readable properties:

Readable streams

A few insights from reading the node/lib/_stream_readable.js source code:

var stream = new stream.Readable();
var state = stream._readableState;

Examples

Custom instance of stream.Transform({objectMode: true}):

{ _readableState:
   { highWaterMark: 16384,
     buffer: [],
     length: 0,
     pipes: null,
     pipesCount: 0,
     flowing: false,
     ended: false,
     endEmitted: false,
     reading: false,
     calledRead: false,
     sync: false,
     needReadable: true,
     emittedReadable: false,
     readableListening: false,
     objectMode: true,
     defaultEncoding: 'utf8',
     ranOut: false,
     awaitDrain: 0,
     readingMore: false,
     decoder: null,
     encoding: null },
  readable: true,
  domain: null,
  _events:
   { end: { [Function: g] listener: [Function: onend] },
     finish: { [Function: g] listener: [Function] } },
  _maxListeners: 10,
  _writableState:
   { highWaterMark: 16384,
     objectMode: true,
     needDrain: false,
     ending: false,
     ended: false,
     finished: false,
     decodeStrings: true,
     defaultEncoding: 'utf8',
     length: 0,
     writing: false,
     sync: true,
     bufferProcessing: false,
     onwrite: [Function],
     writecb: null,
     writelen: 0,
     buffer: [],
     errorEmitted: false },
  writable: true,
  allowHalfOpen: true,
  _transformState:
   { afterTransform: [Function],
     needTransform: false,
     transforming: false,
     writecb: null,
     writechunk: null },
  _transform: [Function] }

Custom instance of stream.Duplex({objectMode: true}):

{ _readableState:
   { highWaterMark: 16384,
     buffer: [],
     length: 0,
     pipes: null,
     pipesCount: 0,
     flowing: false,
     ended: false,
     endEmitted: false,
     reading: false,
     calledRead: false,
     sync: true,
     needReadable: false,
     emittedReadable: false,
     readableListening: false,
     objectMode: true,
     defaultEncoding: 'utf8',
     ranOut: false,
     awaitDrain: 0,
     readingMore: false,
     decoder: null,
     encoding: null },
  readable: true,
  domain: null,
  _events: { end: { [Function: g] listener: [Function: onend] } },
  _maxListeners: 10,
  _writableState:
   { highWaterMark: 16384,
     objectMode: true,
     needDrain: false,
     ending: false,
     ended: false,
     finished: false,
     decodeStrings: true,
     defaultEncoding: 'utf8',
     length: 0,
     writing: false,
     sync: true,
     bufferProcessing: false,
     onwrite: [Function],
     writecb: null,
     writelen: 0,
     buffer: [],
     errorEmitted: false },
  writable: true,
  allowHalfOpen: true }