When handling HTTP responses, especially with streaming data, properly verify content integrity and length to prevent data corruption or unexpected termination. Key considerations:
Content-Length may not match actual data size when responses use compression (gzip, br, deflate). In such cases:
Check for Content-Encoding header to determine if response is compressed
Use actual byte counts rather than Content-Length for progress tracking
Implement proper stream handling based on environment:
For Node.js, use Transform streams rather than ‘data’ event listeners to avoid backpressure issues:
```javascript
// GOOD: Use piping with Transform streams
var contentLengthCheckerStream = new ContentLengthCheckerStream(
parseInt(headers[‘content-length’], 10)
);
responseStream.pipe(contentLengthCheckerStream).pipe(stream);
// AVOID: Direct data event listeners can cause backpressure issues in older Node.js
responseStream.on(‘data’, function(chunk) {
receivedLen += chunk.length;
// potential for data loss in Node.js <= 0.10.x
});
```
Handle content integrity verification for both Node.js and browser environments:
In Node.js, implement pipe-through verification streams
In browsers, buffer response data for integrity checks
Verify checksums/hashes when provided in response headers
Emit appropriate errors with descriptive messages when content length or integrity checks fail
Implementing these practices ensures data is received completely and correctly, preventing subtle bugs in network communication.
