{ "id": "171020", "key": "DAEMON-230", "fields": { "issuetype": { "id": "4", "description": "An improvement or enhancement to an existing feature or task.", "name": "Improvement", "subtask": false }, "project": { "id": "12519", "key": "DAEMON", "name": "Appcelerator Daemon" }, "fixVersions": [], "resolution": { "id": "7", "description": "", "name": "Invalid" }, "resolutiondate": "2018-02-14T23:57:37.000+0000", "created": "2018-02-02T16:42:06.000+0000", "priority": { "name": "Critical", "id": "1" }, "labels": [], "versions": [], "issuelinks": [], "assignee": { "name": "cbarber", "key": "cbarber", "displayName": "Chris Barber", "active": true, "timeZone": "America/Chicago" }, "updated": "2018-02-14T23:57:41.000+0000", "status": { "description": "A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, or are closed.", "name": "Resolved", "id": "5", "statusCategory": { "id": 3, "key": "done", "colorName": "green", "name": "Done" } }, "components": [ { "id": "15050", "name": "appcd-core", "description": "Core daemon logic: config, plugin loader, logging, dispatcher" }, { "id": "15614", "name": "appcd-plugin" } ], "description": "When a plugin sends a JS object response over IPC or a service writes a JS object response over the WebSocket or HTTP response, the JS object is serialized using {{JSON.stringify()}}. However, {{JSON.stringify()}} eats up a ton of memory when the JS object is large. It would be much more efficient to stringify while writing to the stream.\r\n\r\nThere are several npm packages that will stringify a stream, but they are geared towards each write being a small object which they just call {{JSON.stringify()}} on it.\r\n\r\nWhat we need is transforming stream that does stringifies the JS object on the fly. Stringify is not a complex algorithm, but would need to figure out how to handle Node specific objects like {{Buffer}}.", "attachment": [], "flagged": false, "summary": "Stream stringified JSON data over IPC and response instead of JSON.stringify()", "creator": { "name": "cbarber", "key": "cbarber", "displayName": "Chris Barber", "active": true, "timeZone": "America/Chicago" }, "subtasks": [], "reporter": { "name": "cbarber", "key": "cbarber", "displayName": "Chris Barber", "active": true, "timeZone": "America/Chicago" }, "environment": null, "closedSprints": [ { "id": 996, "state": "closed", "name": "2018 Sprint 03 Tooling", "startDate": "2018-01-28T16:24:33.898Z", "endDate": "2018-02-11T16:24:00.000Z", "completeDate": "2018-02-12T04:23:35.987Z", "originBoardId": 219 }, { "id": 1001, "state": "closed", "name": "2018 Sprint 04 Tooling", "startDate": "2018-02-12T04:25:16.870Z", "endDate": "2018-02-26T04:25:00.000Z", "completeDate": "2018-02-26T19:38:13.619Z", "originBoardId": 219 } ], "comment": { "comments": [ { "id": "434501", "author": { "name": "cbarber", "key": "cbarber", "displayName": "Chris Barber", "active": true, "timeZone": "America/Chicago" }, "body": "As good as an idea this is, it's not possible. :(\r\n\r\nWhen a plugin sends a response over IPC to the parent daemon process, it is calling {{process.send()}} which handles the serialization for us and there's no \"streaming\" taking place.\r\n\r\nWhen the daemon sends a response over a WebSocket, the {{ws}} library only has a simple {{ws.send()}} function and no stream interface. Internally it will write the data to a socket which is a stream interface, but it's abstracted by wrapping the data in a frame.\r\n\r\nIt's worth noting that WebSocket responses are encoded with msgpack. msgpack supports streaming, so if {{ws}} should ever support streaming, we could use something like https://www.npmjs.com/package/bfj or https://www.npmjs.com/package/big-json to stringify the response and then stream it through msgpack as we write the bytes over the wire.", "updateAuthor": { "name": "cbarber", "key": "cbarber", "displayName": "Chris Barber", "active": true, "timeZone": "America/Chicago" }, "created": "2018-02-14T23:57:37.000+0000", "updated": "2018-02-14T23:57:37.000+0000" } ], "maxResults": 1, "total": 1, "startAt": 0 } } }