{ "id": "84213", "key": "TIMOB-6849", "fields": { "issuetype": { "id": "1", "description": "A problem which impairs or prevents the functions of the product.", "name": "Bug", "subtask": false }, "project": { "id": "10153", "key": "TIMOB", "name": "Titanium SDK/CLI", "projectCategory": { "id": "10100", "description": "Titanium and related SDKs used in application development", "name": "Client" } }, "fixVersions": [], "resolution": { "id": "7", "description": "", "name": "Invalid" }, "resolutiondate": "2012-03-27T13:47:32.000+0000", "created": "2011-12-22T13:04:21.000+0000", "priority": { "name": "High", "id": "2" }, "labels": [], "versions": [ { "id": "12580", "description": "Dual Runtime 1.8.0", "name": "Release 1.8.0.1", "archived": true, "released": true, "releaseDate": "2011-12-22" } ], "issuelinks": [], "assignee": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "updated": "2014-06-19T12:43:41.000+0000", "status": { "description": "The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.", "name": "Closed", "id": "6", "statusCategory": { "id": 3, "key": "done", "colorName": "green", "name": "Done" } }, "components": [ { "id": "10206", "name": "iOS", "description": "iOS Platform" } ], "description": "h3. Problem\r\n\r\nApplication hangs on parsing Very Lage amount data.\r\n\r\nh3. Reproducible steps:\r\n \r\n1. Run the code below\r\n2. See the console\r\n\r\nNotice that Titanium still stucks on JSON.parse().\r\n\r\nh3. Expected behavior:\r\n\r\nCustomer wants to parse his information.\r\n\r\n\r\nh3. Tested with the next specs:\r\n\r\nSimulator\r\nTitanium Studio, build: 1.0.8.201112191627\r\nTitanium SDK: 1.8.0.1.v20111220190134 , 1.8.0.1.RC3, 1.7.5 & 1.9.0.v20111221123134\r\niOS SDK: 5.0\r\n\r\n{code:title=1.8.0.1.v20111205164258 Titanium CI Build & iOS 5 SDK}\r\n\r\n[INFO] Application started\r\n[INFO] i13399/1.0 (1.8.0.1.v20111220190134.f3575f8)\r\n[INFO] LOGIN REQUEST WAS SENT: http://mobilisticstorage.cloudapp.net/login/spoad@acorndistributors.com/c1AxMjM0\r\n[INFO] LOGIN RESPONSE TEXT: \"53c0f5e4-d9ff-47e8-9196-95f378786e43\"\r\n[INFO] LOGIN RESPONSE OBJECT: 53c0f5e4-d9ff-47e8-9196-95f378786e43\r\n[INFO] GET DATA REQUEST WAS SENT: http://mobilisticstorage.cloudapp.net/dashboard/salesperson?apikey=53c0f5e4-d9ff-47e8-9196-95f378786e43\r\n[INFO] GET DATA RESPONSE TEXT LENGTH: 5439224\r\n{code}\r\n\r\n{code: title=1.8.0.1.RC3 Titanium SDK & iOS 5 SDK}\r\n\r\n[INFO] Application started\r\n[INFO] i13399/1.0 (1.8.0.1.RC3.c903964)\r\n[INFO] LOGIN REQUEST WAS SENT: http://mobilisticstorage.cloudapp.net/login/spoad@acorndistributors.com/c1AxMjM0\r\n[INFO] LOGIN RESPONSE TEXT: \"53c0f5e4-d9ff-47e8-9196-95f378786e43\"\r\n[INFO] LOGIN RESPONSE OBJECT: 53c0f5e4-d9ff-47e8-9196-95f378786e43\r\n[INFO] GET DATA REQUEST WAS SENT: http://mobilisticstorage.cloudapp.net/dashboard/salesperson?apikey=53c0f5e4-d9ff-47e8-9196-95f378786e43\r\n[INFO] GET DATA RESPONSE TEXT LENGTH: 5439224\r\n{code}\r\n\r\n{code: title=1.7.5 Titanium SDK & iOS 5 SDK}\r\n\r\n[INFO] Application started\r\n[INFO] i13399/1.0 (1.7.5.ab20af7)\r\n[INFO] LOGIN REQUEST WAS SENT: http://mobilisticstorage.cloudapp.net/login/spoad@acorndistributors.com/c1AxMjM0\r\n[INFO] LOGIN RESPONSE TEXT: \"53c0f5e4-d9ff-47e8-9196-95f378786e43\"\r\n[INFO] LOGIN RESPONSE OBJECT: 53c0f5e4-d9ff-47e8-9196-95f378786e43\r\n[INFO] GET DATA REQUEST WAS SENT: http://mobilisticstorage.cloudapp.net/dashboard/salesperson?apikey=53c0f5e4-d9ff-47e8-9196-95f378786e43\r\n[INFO] GET DATA RESPONSE TEXT LENGTH: 5439224\r\n{code}\r\n\r\nh3. Helpdesk\r\nAPP-547691 \r\n\r\n \r\n ", "attachment": [], "flagged": false, "summary": "iOS: Application hangs when trying to parse LARGE set of data using JSON.parse()", "creator": { "name": "framirez", "key": "framirez", "displayName": "Francisco Antonio Duran Ramirez", "active": true, "timeZone": "America/Los_Angeles" }, "subtasks": [], "reporter": { "name": "framirez", "key": "framirez", "displayName": "Francisco Antonio Duran Ramirez", "active": true, "timeZone": "America/Los_Angeles" }, "environment": "Simulator\r\nTitanium Studio, build: 1.0.8.201112191627\r\nTitanium SDK: 1.8.0.1.v20111220190134 , 1.8.0.1.RC3, 1.7.5 & 1.9.0.v20111221123134\r\niOS SDK: 5.0", "comment": { "comments": [ { "id": "177069", "author": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "body": "This is an incoming set of 5MB of data - even parsing this on a desktop machine is slow (even with very, very fast specialized JSON parsers), and for mobile devices, it will be even slower. We can upgrade our JSON parser, but for data of this size, it will not keep the application from \"hanging\" while it processes a payload this large.\r\n\r\nThe reason for this is that JSON.parse() is meant to be serialized and synchronous - and Javascript is a serialized and synchronous language. We do have plans to offer threading support in a future release, which will make it possible to perform tasks like JSON deserialization as a background task, \r\n\r\nJSON is generally intended to be used for relatively lightweight datagrams (certainly under 5MB). I'm unsure what the intention behind such a large payload is, but there are some other options for processing:\r\n\r\n* Retrieve specific data as requested via SQL queries\r\n* Cache information locally after an initial retrieval\r\n* Split data into smaller payloads that can be processed on an \"as-needed\" basis\r\n\r\nIf this data must be processed in a complete chunk and cannot be divided in any way, the following is recommended: \r\n\r\n* Present an activity indicator in a window before JSON begins loading, and lock UI activity\r\n* Parse the JSON\r\n* Hide the activity indicator and unlock UI\r\n\r\nThis prevents your app from appearing unresponsive while the data is processed.", "updateAuthor": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2011-12-22T15:21:05.000+0000", "updated": "2011-12-22T15:21:05.000+0000" }, { "id": "177147", "author": { "name": "a.konovkov", "key": "a.konovkov", "displayName": "Alexander Konovkov", "active": true, "timeZone": "America/Los_Angeles" }, "body": "Stephen, actually it parses such large data ok, but it fails to iterate through it with four `indexOf` function calls on each iteration.", "updateAuthor": { "name": "a.konovkov", "key": "a.konovkov", "displayName": "Alexander Konovkov", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2011-12-24T07:23:17.000+0000", "updated": "2011-12-24T07:23:17.000+0000" }, { "id": "177341", "author": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "body": "Alexander -\r\n\r\nSorry, I missed part of the initial bug report. This could be an issue having nothing to do with JSON data but instead the ability to iterate over the parsed object due to its unusually large size. We will be taking a closer look at the problem.", "updateAuthor": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2011-12-27T16:28:46.000+0000", "updated": "2011-12-27T16:28:46.000+0000" }, { "id": "177449", "author": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "body": "Deleted an earlier comment with bad source code. The following works:\r\n\r\n{code}\r\nvar jsonStr = '{\"foo\":\"bar\"}';\r\nvar jsonObj = {\"foo\":\"bar\"};\r\n\r\nvar xhr = Ti.Network.createHTTPClient();\r\nxhr.onload = function(e) {\r\n\tTi.API.info(\"BEFORE PARSE: \"+jsonStr);\r\n//\tvar obj = null;\r\n\tvar obj = JSON.parse(jsonStr);\r\n\tTi.API.info(\"AFTER PARSE: \"+obj);\r\n};\r\nTi.API.info(\"obj->str: \"+JSON.stringify(jsonObj));\r\nTi.API.info(\"Outside onload: \"+JSON.parse(jsonStr));\r\nxhr.open('GET', 'http://appcelerator.com');\r\nxhr.send();\r\n{code}\r\n\r\nNote that if we change the {{jsonStr}} to be a bad JSON value (such as {{\"{foo:bar}\"}}) then we throw an exception in the callback which is not properly handled. This leads me to believe two things:\r\n\r\n* The 5.2 MB JSON contains an error\r\n* We need to properly log exceptions during all callbacks and display them in a reasonable fashion", "updateAuthor": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2011-12-28T12:14:59.000+0000", "updated": "2011-12-28T12:14:59.000+0000" }, { "id": "177494", "author": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "body": "I'm currently running jsonlint on the information dump to see if it contains any errors... and it's been running for 40 minutes and still isn't done. I'm going to run it through a specialized parsing/validation tool, but do not expect it to run any faster.\r\n\r\nThis is on a 2GHz Intel Core i7; significantly faster than any mobile device.\r\n\r\nThis ticket may be marked as INVALID based on the results of a specialized parser run. It's also worth noting that we would have to upgrade to a new version of JSCore with completely rewritten JSON processing internals to make any speed improvements.", "updateAuthor": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2011-12-28T16:05:33.000+0000", "updated": "2011-12-28T16:05:33.000+0000" }, { "id": "177506", "author": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "body": "After running for over an hour, jsonlint produced the following error:\r\n\r\n{code}\r\nError: Parse error on line 1:\r\n...iseTotal\":21.7111000\r\n-----------------------^\r\nExpecting '}', ','\r\n{code}\r\n\r\nEither it ran out of buffer space (possible) or there is an actual JSON error.", "updateAuthor": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2011-12-28T16:55:08.000+0000", "updated": "2011-12-28T16:55:08.000+0000" }, { "id": "177512", "author": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "body": "Resolving as invalid as per issues with the JSON payload and amount of processing time required to handle it under any circumstances.\r\n\r\nIt is worth noting that most libraries which ship with a test for a \"large\" JSON payload consider such a payload to be around 9kb.", "updateAuthor": { "name": "stephentramer", "key": "stephentramer", "displayName": "Stephen Tramer", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2011-12-28T17:16:33.000+0000", "updated": "2011-12-28T17:16:33.000+0000" }, { "id": "180205", "author": { "name": "thomashuelbert", "key": "thomashuelbert", "displayName": "Thomas Huelbert", "active": true, "timeZone": "America/Los_Angeles" }, "body": "closing based on Stevens comments", "updateAuthor": { "name": "thomashuelbert", "key": "thomashuelbert", "displayName": "Thomas Huelbert", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2012-01-23T11:49:53.000+0000", "updated": "2012-01-23T11:49:53.000+0000" } ], "maxResults": 12, "total": 12, "startAt": 0 } } }