Titanium JIRA Archive
Titanium SDK/CLI (TIMOB)

[TIMOB-9111] TiAPI: Ti.Utils.base64encode behaves different between Android and iOS/Android

GitHub Issuen/a
TypeBug
PriorityHigh
StatusClosed
ResolutionFixed
Resolution Date2017-11-15T20:47:31.000+0000
Affected Version/sRelease 2.0.1, Release 3.1.0, Release 3.2.0
Fix Version/sRelease 7.0.0
ComponentsiOS
LabelsSupportTeam, core, parity
ReporterVarun Joshi
AssigneeHans Knöchel
Created2012-05-11T12:28:30.000+0000
Updated2017-11-17T21:45:59.000+0000

Description

Problem

The result of Ti.Utils.base64encode is different between iOS and Android. On iOS there are newline characters included when the string is long enough, on Android the newline characters are stripped out (which is the correct way to do base64 UTF8 encoding). Please see the attached screenshot explaining the issue.

Sample Code

{noformat} (function() { var win = Ti.UI.createWindow({backgroundColor:'white'}); win.open(); var shortString = 'ABCDEFGHIJ1234567890ABCDEFGHIJ12|psndemo2|abcd:1'; var longString = 'ABCDEFGHIJ1234567890ABCDEFGHIJ12|psndemo2|abcd:12345678901234567890'; var tiBase64ShortResult = Ti.Utils.base64encode(shortString); var tiBase64LongResult = Ti.Utils.base64encode(longString); tiBase64ShortResult = String(tiBase64ShortResult); tiBase64LongResult = String(tiBase64LongResult); // BUG #2: Run this code in iOS and Android and compare the results // On iOS we get newline characters. On Android the newline characters are stripped out. // The correct behavior for UTF8 base64 encoding is to strip out the newline characters. // Android's implementation is correct. tiBase64ShortResult = tiBase64ShortResult.replace(/\r\n/g,"***NEWLINE***"); tiBase64LongResult = tiBase64LongResult.replace(/\r\n/g,"***NEWLINE***"); var alertDialog = Titanium.UI.createAlertDialog( { title: 'Result', message: 'short: ' + tiBase64ShortResult + '\nlong: ' + tiBase64LongResult, buttonNames: ['OK','Cancel'] }); alertDialog.show(); })(); {noformat}

Attachments

FileDateSize
encodedFile.txt2017-11-01T16:35:09.000+000029
Screen shot 2012-05-08 at 3.43.47 PM.png2012-05-11T12:28:30.000+0000690743

Comments

  1. Shalvika 2012-10-29

    Could be because of the problem mentioned here http://developer.appcelerator.com/question/120731/xhr-authentication-with-restful-api "It's also important to note that Titanium.Utils.base64encode() adds line breaks every 72 characters (bug?)"
  2. jithinpv 2013-04-12

    Issue reproduces Tested with Titanium Studio, build: 3.0.1.201212181159 Titanium SDK version: 3.1.0 Titanium SDK version: 3.0.2 iOS iPhone Simulator: iOS SDK version: 6.0
  3. Fokke Zandbergen 2016-04-07

    Can we get this scheduled please? After verifying newlines indeed shouldn't be included in base64encoded data, all we need to do is remove these here: https://github.com/appcelerator/titanium_mobile/blob/415bd6c66dcc55b1a59a59574f3babd3c3a84ede/iphone/Classes/Base64Transcoder.c#L132-L136 Also see: * http://stackoverflow.com/a/20065991/4626813 * http://stackoverflow.com/a/4680044/4626813 * https://en.wikipedia.org/wiki/Base64#Implementations_and_history Seems like it depends on the implementation. Perhaps we should make it an option.
  4. Ryan Gartin 2016-04-08

    BUMP! I've wasted 8+ hours only to find out this is the issue! HALP!
  5. Angel Petkov 2016-04-27

    PR: https://github.com/appcelerator/titanium_mobile/pull/7976 Demo Code: 
       var win = Ti.UI.createWindow({backgroundColor:'white'});
   var shortString = 'ABCDEFGHIJ1234567890ABCDEFGHIJ12|psndemo2|abcd:1';
   var longString  = 'ABCDEFGHIJ1234567890ABCDEFGHIJ12|psndemo2|abcd:12345678901234567890';
   var tiBase64ShortResult = Ti.Utils.base64encode(shortString);
   var tiBase64LongResult  = Ti.Utils.base64encode(longString);
   
   tiBase64ShortResult = String(tiBase64ShortResult);
   tiBase64LongResult = String(tiBase64LongResult);
   
   var label2 = Ti.UI.createLabel({
     color:'blue',
     text: 'short: ' + tiBase64ShortResult + '\nlong: ' + tiBase64LongResult,
     textAlign: Ti.UI.TEXT_ALIGNMENT_LEFT,
     top: 30,
     width: 300, height: 200
   });
   win.add(label2);
   win.open()
  6. Hieu Pham 2016-05-25

    What is the expected value of the demo above? And are the results going to be the same on both Android and iOS?
  7. Angel Petkov 2016-05-25

    There should just be no more new lines added redundantly, yes I believe they should be. There is more information available regarding the return values on the PR.
  8. Hieu Pham 2016-05-25

    I'm saying that you should include the expected value inside the demo so when QE test it, they know what to expect, and if somehow they get a different value, then they would know its a bug.
  9. Chee Kiat Ng 2016-06-02

    [~apetkov] What's the status of this?
  10. Christopher Williams 2016-06-09

    I posted some discussion about this on the PR, but I wanted to summarize here: - I think if anything, the fix should be to Android to default to adding newlines to split the output up. There are a number of uses/specs for base64: - https://tools.ietf.org/html/rfc4648#page-3 - https://en.wikipedia.org/wiki/Base64#Variants_summary_table Some of those specs *require* max line lengths (specifically 76 or 64 characters max per line). iOS and Windows both default to adding newlines to break up the output. So really, Android is the odd one out in comparison. Or, I suppose we can choose to default to no newlines, but would need to change both iOS and Windows. - I think we should modify the base64 method to take a second optional Dictionary argument to control behavior like this. For this who need a specific line length, or want to turn off newlines, or use different padding characters or characters for indices 62/63 - so that they could match the listed specs at https://en.wikipedia.org/wiki/Base64#Variants_summary_table Android's base64 does have some options around some of this: https://developer.android.com/reference/android/util/Base64.html Basically it looks like the defaults for encoding should hopefully be the options that'd work across most applications: max 76-char lines, use CRLF to break lines, use '=' for padding character, use '+' and '/' for indices 62 and 63. Then we could add options for: no newlines, URL safe chars ('-' and '_' for 62/63), etc.
  11. Hans Knöchel 2017-08-08

    Moving to 7.0.0 since this will be a (minor) breaking change on iOS and Windows. We will remove the \n characters to align with modern conversions. It will also eliminate possible issues with using those strings in the web-view (especially Ti.WKWebView).
  12. Hans Knöchel 2017-08-08

    PR (titanium_verify/master): https://github.com/appcelerator/titanium_verify/pull/25 PR (titanium_mobile/master): https://github.com/appcelerator/titanium_mobile/pull/9288 Test-case (encodedFile.txt is attached: 
        var win = Ti.UI.createWindow({
        backgroundColor: '#fff'
    });
    
    var btn = Ti.UI.createButton({
        title: 'Trigger'
    });
    
    btn.addEventListener('click', function() {
        Ti.API.info('--- ENCODE! --- ');
        
        var clearText = 'test';
        var encoded = 'dGVzdA==';
        
        var test = Ti.Utils.base64encode(clearText);
        
        if (test.apiName !== 'Ti.Blob') {
          Ti.API.error('test is not a Ti.Blob');
          Ti.API.error(test.apiName);
        } else {
          Ti.API.warn('Test 1/5 passed');
        }
        
        if (test.getText() !== encoded) {
          Ti.API.error('test is not properly encoded');
          Ti.API.error(test.getText());
        } else {
          Ti.API.warn('Test 2/5 passed');
        }
        
        Ti.API.info('--- DECODE! --- ');
        
        test = Ti.Utils.base64decode(encoded);
    
        if (test.apiName !== 'Ti.Blob') {
          Ti.API.error('test is not a Ti.Blob');
          Ti.API.error(test.apiName);
        } else {
          Ti.API.warn('Test 3/5 passed');
        }
    
        if (test.getText() !== clearText) {
          Ti.API.error('test is not properly decoded');
          Ti.API.error(test.getText());
        } else {
          Ti.API.warn('Test 4/5 passed');
        }
        
        var f = Titanium.Filesystem.getFile(Titanium.Filesystem.resourcesDirectory, 'encodedFile.txt');
        var blob = Ti.Utils.base64decode(f.read());
            
        if (blob.toString() !== 'Decoding successful!') {
          Ti.API.error('Blob-decode not working!');
          Ti.API.error(blob.toString());
        } else {
          Ti.API.warn('Test 5/5 passed');
        }
    });
    
    win.add(btn);
    win.open();
  13. Kota Iguchi 2017-11-15

    Cloned this for Windows: TIMOB-25513 with fixVersion:none since I don't think we can slip this in to Windows within 7.0.0 time frame.
  14. Abir Mukherjee 2017-11-15

    FR passed. [~kiguchi] thanks for creating the ticket for Windows. Currently looking into unit test issues.
  15. Eric Wieber 2017-11-17

    Verified in SDK build 7.0.0.v20171117124508

JSON Source