[TIMOB-1719] iOS: non-UTF-8 data from external source cannot be saved and reopened accurately

GitHub Issuen/a
ResolutionWon't Fix
Resolution Date2017-07-26T03:23:04.000+0000
Affected Version/sRelease 3.0.0
Fix Version/sn/a
ReporterThomas Huelbert
AssigneeIngo Muschenetz


This was addressed on Android, over to iOS for parity. Below are the notes Bill wrote in the original bug. From #1690

Use case (helpdesk 38891): app fetches data via XHR. Data is XML encoded in ISO-8859-1. Data is saved to file, to be opened later. When re-opened, Titanium forces it to UTF-8, which screws up the encoding of some of the special characters such as those with umlaute (ö, ä etc) or accents (é etc).

Fail case:

script to repro:

var win = Titanium.UI.createWindow({

title:'Feed encoding test',
fullscreen: true,
exitOnClose: true,
url: 'main.js'

}); win.open();

var win = Ti.UI.currentWindow;
win.title = "Feed encoding via file"
var url = [SEE HELPDESK 38891]

var dir = Titanium.Filesystem.applicationDataDirectory;
var filename = 'feed.xml';
var file = Ti.Filesystem.getFile(dir, filename);

function saveFeed(data) {

if (file.exists()){


function buildTable() {

var blob = file.read();
var string = blob.text;
var xml = Ti.XML.parseString(string);
var rows = [];
try {
    if (xml) {
        var session = xml.getElementsByTagName('session');
        if (session) {
            session = session.item(0);
        } else { 
            alert('session list not fetched');
        if (session) {
            if (!session.hasChildNodes()) {
                alert('No child nodes');
            var length = session.childNodes.length;
            for (var i = 0; i < length; i++) {
                var child = session.childNodes.item(i);
                if (child.nodeType == child.ELEMENT_NODE) {
                    rows.push(Ti.UI.createTableViewRow({color: 'black', title: child.getAttribute("driver")}));
            win.add(Ti.UI.createTableView({data: rows}));
        } else {
            alert('"session" not found');
    } else {
        alert('XML did not load');
} catch(ex) {


var xhr = Ti.Network.createHTTPClient();
xhr.onload = function(e) {

try {
    var data = xhr.responseData;

} catch(ex) {

}; xhr.open('GET', url);


  1. Thomas Huelbert 2011-04-15

    more of Bills notes:
    We want to keep things internally in UTF-8, so instead of trying to track the other encoding throughout its lifetime in Titanium code (i.e., in TiBlob, TiFile, etc), we now give the developer a way to transcode the text to utf-8 and then save that:
    var utf8Text = Ti.Utils.transcodeString(origText, 'ISO-8859-1', 'UTF-8');
    So Ti.Utils.transcodeString(origText, origEncoding, desiredEncoding) is new.
    To fix the fail case from the description of this item, in the function saveFeed change file.write(data) to file.write(Ti.Utils.transcodeString(data, 'ISO-8859-1', 'UTF-8')).
    Then, because the feed contains <?xml version="1.0" encoding="ISO-8859-1"?>, you need to change the encoding= to read encoding='UTF-8 before giving it to the xml parser. So in buildTable() change ...
    var xml = Ti.XML.parseString(string);
    ... to ... var xml = Ti.XML.parseString(string.replace(/iso-8859-1/i, 'UTF-8'));
    Finally, when the data comes back from the provider, get it with responseText instead of responseData
    . So replace this... var data = xhr.responseData;
    with this ...

    var data = xhr.responseText;


    xhr.responseData with xhr.responseText

  2. Thomas Huelbert 2011-04-15

    the current behavior is the test generates an alert on iOS. I posted the working test case (resource files) to Appcelerator/Users/Thomas/bugs/1719 if that helpful

  3. Bill Dawson 2011-04-15

    We added Ti.Utils.transcodeString, so Ti iOS should get one too for parity. Here's the java, fyi:

       public String transcodeString(String orig, String inEncoding, String outEncoding)
               try {
                   Charset charsetOut = Charset.forName(outEncoding);
                   Charset charsetIn = Charset.forName(inEncoding);
                   ByteBuffer bufferIn = ByteBuffer.wrap(orig.getBytes(charsetIn.name()) );
                   CharBuffer dataIn = charsetIn.decode(bufferIn);
                   bufferIn = null;
                   ByteBuffer bufferOut = charsetOut.encode(dataIn);
                   dataIn = null;
                   byte[] dataOut = bufferOut.array();
                   bufferOut = null;
                   return new String(dataOut, charsetOut.name());
               } catch (UnsupportedEncodingException e) {
                   Log.e(LCAT, "Unsupported encoding: " + e.getMessage(), e);
               return null;
  4. Blain Hamon 2011-04-15

    We could leave this as an entrance test for new hires. Very clean and clearcut function.

  5. Stephen Tramer 2012-07-25 Despite the lack of a working existing test case (no URL provided in test) it is clear that this bug is still valid in SDK because we have NO UTF-16 support. Options are:

    Provide an alternative test from original customer

    Find a UTF-16 datastream to test against

    Tagging for core as it is part of a broader UTF-16 support push.
  6. Lee Morris 2017-07-26 Closing due to inactivity. If this issue still exists, please raise a new ticket.

JSON Source