{ "id": "62322", "key": "TIMOB-1690", "fields": { "issuetype": { "id": "1", "description": "A problem which impairs or prevents the functions of the product.", "name": "Bug", "subtask": false }, "project": { "id": "10153", "key": "TIMOB", "name": "Titanium SDK/CLI", "projectCategory": { "id": "10100", "description": "Titanium and related SDKs used in application development", "name": "Client" } }, "fixVersions": [ { "id": "11225", "name": "Release 1.5.0", "archived": true, "released": true, "releaseDate": "2010-12-14" } ], "resolution": { "id": "1", "description": "A fix for this issue is checked into the tree and tested.", "name": "Fixed" }, "resolutiondate": "2011-04-17T01:56:53.000+0000", "created": "2011-04-15T02:59:27.000+0000", "priority": { "name": "Trivial", "id": "5" }, "labels": [ "android", "defect" ], "versions": [], "issuelinks": [], "assignee": { "name": "billdawson", "key": "billdawson", "displayName": "Bill Dawson", "active": true, "timeZone": "Europe/Berlin" }, "updated": "2011-04-17T01:56:53.000+0000", "status": { "description": "The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.", "name": "Closed", "id": "6", "statusCategory": { "id": 3, "key": "done", "colorName": "green", "name": "Done" } }, "components": [ { "id": "10202", "name": "Android", "description": "Android Platform" } ], "description": "{html}

Use case (helpdesk 38891): app fetches data via XHR. Data is XML\nencoded in ISO-8859-1. Data is saved to file, to be opened later.\nWhen re-opened, Titanium forces it to UTF-8, which screws up the\nencoding of some of the special characters such as those with\numlaute (ö, ä etc) or accents (é etc).

\n

Fail case:

\n

app.js

\n
\n\nTitanium.UI.setBackgroundColor('#000');\nvar win = Titanium.UI.createWindow({  \n    title:'Feed encoding test',\n    backgroundColor:'#fff',\n    fullscreen: true,\n    exitOnClose: true,\n    url: 'main.js'\n});\nwin.open();\n
\n

main.js:

\n
\nvar win = Ti.UI.currentWindow;\nwin.title = \"Feed encoding via file\"\nvar url = [SEE HELPDESK 38891]\n\nvar dir = Titanium.Filesystem.applicationDataDirectory;\nvar filename = 'feed.xml';\nvar file = Ti.Filesystem.getFile(dir, filename);\n\n\nfunction saveFeed(data) {\n    if (file.exists()){\n        file.deleteFile();\n    }\n    file.write(data);\n}\n\nfunction buildTable() {\n    var blob = file.read();\n    var string = blob.text;\n    var xml = Ti.XML.parseString(string);\n    var rows = [];\n    try {\n        if (xml) {\n            var session = xml.getElementsByTagName('session');\n            if (session) {\n                session = session.item(0);\n            } else { \n                alert('session list not fetched');\n                return;\n            }\n            if (session) {\n                if (!session.hasChildNodes()) {\n                    alert('No child nodes');\n                    return;\n                }\n                var length = session.childNodes.length;\n                for (var i = 0; i < length; i++) {\n                    var child = session.childNodes.item(i);\n                    if (child.nodeType == child.ELEMENT_NODE) {\n                        rows.push(Ti.UI.createTableViewRow({color: 'black', title: child.getAttribute(\"driver\")}));\n                    }\n                }\n                win.add(Ti.UI.createTableView({data: rows}));\n            } else {\n                alert('\"session\" not found');\n            }\n        } else {\n            alert('XML did not load');\n        }\n    } catch(ex) {\n        alert(ex);\n    }\n    \n}\n\nvar xhr = Ti.Network.createHTTPClient();\nxhr.onload = function(e) {\n    try {\n        var data = xhr.responseData;\n        saveFeed(data);\n        buildTable();\n        \n    } catch(ex) {\n        alert(ex);\n    }\n};\nxhr.open('GET', url);\nxhr.send();\n
\n

In the tableview that results from running that code, you will\nnotice some of the names have characters that are messed up.

{html}", "attachment": [], "flagged": false, "summary": "Android: non-UTF-8 data from external source cannot be saved and reopened accurately", "creator": { "name": "billdawson", "key": "billdawson", "displayName": "Bill Dawson", "active": true, "timeZone": "Europe/Berlin" }, "subtasks": [], "reporter": { "name": "billdawson", "key": "billdawson", "displayName": "Bill Dawson", "active": true, "timeZone": "Europe/Berlin" }, "environment": null, "comment": { "comments": [ { "id": "126368", "author": { "name": "jhaynie", "key": "jhaynie", "displayName": "Jeff Haynie", "active": false, "timeZone": "America/Los_Angeles" }, "body": "{html}

(from [32cf760dca3156a8478fa47e519070c900a32fab])\n[#1690] Put a transcodeString() utility function\ninto UtilsModule. \nhttp://github.com/appcelerator/titanium_mobile/commit/32cf760dca315...

{html}", "updateAuthor": { "name": "jhaynie", "key": "jhaynie", "displayName": "Jeff Haynie", "active": false, "timeZone": "America/Los_Angeles" }, "created": "2011-04-15T02:59:27.000+0000", "updated": "2011-04-15T02:59:27.000+0000" }, { "id": "126369", "author": { "name": "billdawson", "key": "billdawson", "displayName": "Bill Dawson", "active": true, "timeZone": "Europe/Berlin" }, "body": "{html}

We want to keep things internally in UTF-8, so instead of trying\nto track the other encoding throughout its lifetime in Titanium\ncode (i.e., in TiBlob, TiFile, etc), we now give the developer a\nway to transcode the text to utf-8 and then save that:

\n
\nvar utf8Text = Ti.Utils.transcodeString(origText, 'ISO-8859-1', 'UTF-8');\n
\n

So Ti.Utils.transcodeString(origText, origEncoding,\ndesiredEncoding) is new.

\n

To fix the fail case from the description of this item, in the\nfunction saveFeed change file.write(data)\nto file.write(Ti.Utils.transcodeString(data, 'ISO-8859-1',\n'UTF-8')).

\n

Then, because the feed contains <?xml version=\"1.0\"\nencoding=\"ISO-8859-1\"?>, you need to change the\nencoding= to read encoding='UTF-8 before\ngiving it to the xml parser. So in buildTable() change\n...

\n
\nvar xml = Ti.XML.parseString(string);\n
\n

... to ...

\n
\nvar xml = Ti.XML.parseString(string.replace(/iso-8859-1/i, 'UTF-8'));\n
\n

Finally, when the data comes back from the provider, get it with\nresponseText instead of responseData
\n. So replace this...

\n
\nvar data = xhr.responseData;\n
\n

with this ...

\n
\n\nvar data = xhr.responseText;\n
{html}", "updateAuthor": { "name": "billdawson", "key": "billdawson", "displayName": "Bill Dawson", "active": true, "timeZone": "Europe/Berlin" }, "created": "2011-04-15T02:59:28.000+0000", "updated": "2011-04-15T02:59:28.000+0000" }, { "id": "126370", "author": { "name": "billdawson", "key": "billdawson", "displayName": "Bill Dawson", "active": true, "timeZone": "Europe/Berlin" }, "body": "{html}

I screwed up in the previous comment. I forgot to mention you\nneed to also replace xhr.responseData with\nxhr.responseText. I edited the comment above to\nreflect this now.

{html}", "updateAuthor": { "name": "billdawson", "key": "billdawson", "displayName": "Bill Dawson", "active": true, "timeZone": "Europe/Berlin" }, "created": "2011-04-15T02:59:28.000+0000", "updated": "2011-04-15T02:59:28.000+0000" }, { "id": "126371", "author": { "name": "thomashuelbert", "key": "thomashuelbert", "displayName": "Thomas Huelbert", "active": true, "timeZone": "America/Los_Angeles" }, "body": "{html}

w00t - that did it, thanks Much Bill. Confirmed on nexus 1\nrunning 2.2 and simulator.

{html}", "updateAuthor": { "name": "thomashuelbert", "key": "thomashuelbert", "displayName": "Thomas Huelbert", "active": true, "timeZone": "America/Los_Angeles" }, "created": "2011-04-15T02:59:28.000+0000", "updated": "2011-04-15T02:59:28.000+0000" } ], "maxResults": 4, "total": 4, "startAt": 0 } } }