Titanium JIRA Archive
Titanium SDK/CLI (TIMOB)

[TIMOB-23183] iOS Non-public API usage: The app references non-public symbols in <APPLICATION NAME> : _ptrace

GitHub Issuen/a
TypeBug
PriorityCritical
StatusClosed
ResolutionFixed
Resolution Date2018-02-26T22:06:20.000+0000
Affected Version/sn/a
Fix Version/sRelease 7.3.0
ComponentsiOS
Labelsios
Reporterngweixing
AssigneeEric Wieber
Created2016-03-31T14:58:45.000+0000
Updated2018-07-11T20:09:50.000+0000

Description

Comments

  1. Sharif AbuDarda 2016-04-06

    Hello, There is a similar reported issue in the stake overflow. But that seems to be for the native platform. http://stackoverflow.com/questions/21829521/fails-to-distribute-my-app-your-app-contains-non-public-api-usage. Can you try following the ticket. Regards, Sharif
  2. ngweixing 2016-04-06

    Hi, If you have read my post above carefully, you would have noticed the difference between your link and our issue. The ptrace problem was due to the fact that version 5.2.1 has a fix for the appc-security-debugger-detect crash on ios 9.1. You can search in your jira where there is that bug. So we believe your team tried to tix that problem while introducing another bug in the process. It seems that appcelerator is not really willing to check such a major problem where it impact app submission to the Ios App Store because we have submitted ticket for so many days and no one bother to care about it. It is quite a disappointment really.
  3. Chee Kiat Ng 2016-04-07

  4. ngweixing 2016-04-08

    Hi, We already used the previous version to compile and submit to app store. We will try the latest release very soon. By the way, did your team use "ptrace" in your code in order to detect users connecting debugger tool? And thanks for giving us something related to the issue which could possibly be the solution. Thanks, Heng
  5. Peter Ladis 2016-04-10

    I just got this error: The app references non-public symbols in LilyPad: _ptrace from iTunes when using GA 5.2.2. Any suggestions?
  6. Chee Kiat Ng 2016-04-11

    [~peterladis] May I know if you configured this in your tiapp.xml?
         <property name="appc-sourcecode-encryption-policy">remote</property>
         <property name="appc-security-debugger-detect">true</property>
         <property name="appc-security-jailbreak-detect">true</property>  
       
    and if removing all 3 properties helps? [~ngweixing] may I ask if you faced the same issue with all 3 properties above removed on 5.2.1.GA? Thanks.
  7. ngweixing 2016-04-11

    Hi @cheekiatng, with or without the 3 properties also has the ptrace problem. Even with just appc-security-debugger-detect, it also has the error. I haven't tested with version 5.2.2 yet though.
  8. Chee Kiat Ng 2016-04-11

    [~ngweixing] Thanks, we will continue to investigate. If it's related to ptrace, there indeed 1 line of code that may include that, but there's been no changes to it between sdk versions so I'm not sure why it would come up now.
  9. ngweixing 2016-04-11

    Hi kiat, if you want to do a quick test. We try using sublime text editor and search through the build folder in the titanium app folder which contains the xcode project files. Sublime was able to detect the presence of ptrace in the binary somehow. When we do the same using sdk 5.2.0, no ptrace string was found. As long as ptrace is found, apple will reject it. you can also test upload to your testflight via ituneconnect
  10. Chee Kiat Ng 2016-04-11

    ok our team just did a quick test. This only happens if this property is existing in tiapp.xml (no matter if value is remote or embed):
         <property name="appc-sourcecode-encryption-policy">remote</property>
        
    So it looks like our jaibreak detection is broken. We will try to fix this. Meanwhile, the workaround is to not have any of these 3 properties in your tiapp.xml.
  11. ngweixing 2016-04-11

    Thanks for the update. Finally, your team can recognise that there is a problem and is an independent issue from the issue posted on stackoverflow since last year. For sdk 5.2.1, no matter you include any of those 3 properties or not, build always contain ptrace with failed submission to app store. I assume sdk 5.2.2 is what you are experiencing which is the same behaviour on sdk 5.2.0 in which as long as we do not include any of those properties, it submits fine. We really hope you can fix this problem soon for everyone's benefit and for those who have been using these properties to protect their codes and is in urgent needs to upload a new version to the app store.
  12. Chee Kiat Ng 2016-04-25

    [~peterladis] [~ngweixing] Based on our investigations so far, ptrace is only used for detecting debugger, so since our primary concern is detecting jailbreak, please let me know if setting the following properties work for you while we continue to find alternative solutions to this problem.
          <property name="appc-sourcecode-encryption-policy">remote</property>
          <property name="appc-security-debugger-detect">false</property>
          <property name="appc-security-jailbreak-detect">true</property> 
        
  13. ngweixing 2016-04-28

    @Chee Kiat Ng Actually our primary concern is getting debugger detect to work and not jailbreak detect. We don't need to detect jailbreak. We want to use debugger detect to protect our code and api keys in the code, etc
  14. Chee Kiat Ng 2016-04-28

    Hmm. At the moment we still are unable to find a better way to do debugger detect without the use of ptrace. Will keep you posted see what we can do.
  15. ngweixing 2016-04-28

    Ok sure. Thanks for this!
  16. ngweixing 2016-04-29

    Hi Kiat, appcelerator is seriously facing big issues with these parameters. We tested with the latest 5.2.2.GA and still fails to submit to testflight because of ptrace. We placed all 3 parameters you suggest from your previous post. May I know if we don't specify remote for appc-sourcecode-encryption-policy, what is the default value? Now we just want to ensure encryption policy is remote and set the other 2 to false and still get ptrace problem with 5.2.2.GA.
  17. Chee Kiat Ng 2016-04-29

    Don't specify these 3 properties at all, and it'll default to our non-remote encryption. With no jailbreak or debugger detection enabled. Ptrace will not be a problem because the library that has it won't be included [~ngweixing]
  18. Michael Sharman 2016-05-17

    I'm also getting this problem with 5.2.2.GA - I'm not specifying any of those 3 properties. Its currently stopping me upload a update to the AppStore?
  19. Michael Sharman 2016-05-17

    Looks like appc-security-debugger-detect is enabled by default when building for distribution. But even by explicitly turning this off is not enough to remove the _ptrace symbol from the application binary.
  20. Chee Kiat Ng 2016-05-18

    [~michael@thecodesharman.com.au] That's an interesting problem you have there. I'm quite sure the debugger detect is not enabled by default. Do you think you can provide me with more details? Are you using any external modules or libraries?
  21. Michael Sharman 2016-05-18

    No external modules or libraries, I've been trying to figure out where the _ptrace symbol is coming from by passing the using the "-why_live _ptrace" option to ld, this is the result: _ptrace from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS9.3.sdk/usr/lib/libSystem.tbd _icucfasf7797nnzz from /Users/msharman/Documents/walta/walta-app/build/iphone/lib/libtiverify.a(TiVerify.o) +[ApplicationRouting initialize] from /Users/msharman/Documents/walta/walta-app/build/iphone/build/Intermediates/Waterbug.build/Release-iphoneos/Waterbug.build/Objects-normal/arm64/ApplicationRouting.o l_OBJC_$_CLASS_METHODS_ApplicationRouting from /Users/msharman/Documents/walta/walta-app/build/iphone/build/Intermediates/Waterbug.build/Release-iphoneos/Waterbug.build/Objects-normal/arm64/ApplicationRouting.o l_OBJC_METACLASS_RO_$_ApplicationRouting from /Users/msharman/Documents/walta/walta-app/build/iphone/build/Intermediates/Waterbug.build/Release-iphoneos/Waterbug.build/Objects-normal/arm64/ApplicationRouting.o _OBJC_METACLASS_$_ApplicationRouting from /Users/msharman/Documents/walta/walta-app/build/iphone/build/Intermediates/Waterbug.build/Release-iphoneos/Waterbug.build/Objects-normal/arm64/ApplicationRouting.o _OBJC_CLASS_$_ApplicationRouting from /Users/msharman/Documents/walta/walta-app/build/iphone/build/Intermediates/Waterbug.build/Release-iphoneos/Waterbug.build/Objects-normal/arm64/ApplicationRouting.o ltmp10 from /Users/msharman/Documents/walta/walta-app/build/iphone/build/Intermediates/Waterbug.build/Release-iphoneos/Waterbug.build/Objects-normal/arm64/ApplicationRouting.o ltmp10 from /Users/msharman/Documents/walta/walta-app/build/iphone/build/Intermediates/Waterbug.build/Release-iphoneos/Waterbug.build/Objects-normal/arm64/ApplicationRouting.o So it looks like it is being linked in from the ApplicationRouting.m code via the libtiverify.o library regardless of any settings.
  22. Michael Sharman 2016-05-18

    Also I'm using the appcelerator CLI version 5.3.0-43 for this. I note that the libtiverify.a is symlinked to /Users/msharman/.appcelerator/install/5.3.0-43/package/node_modules/appc-cli-titanium/support/ios/libappcverify.a so this may be related to specifically the appcelerator CLI version ?
  23. Chee Kiat Ng 2016-05-18

    [~michael@thecodesharman.com.au] Do you mind sharing the ApplicationRouting.m content? It's in your build/iphone/Classes/
  24. Chee Kiat Ng 2016-05-18

    Thanks for pointing that out [~michael@thecodesharman.com.au]. Those are actually 2 different libraries (libtiverify.a vs libappcverify.a). Are you sure they are symlinked?
  25. Michael Sharman 2016-05-18

    Sure this is the content:
        /**
         * This code is closed source and Confidential and Proprietary to
         * Appcelerator, Inc. All Rights Reserved.  This code MUST not be
         * modified, copied or otherwise redistributed without express
         * written permission of Appcelerator. This file is licensed as
         * part of the Appcelerator Platform and governed under the terms
         * of the Appcelerator license agreement.
         * Copyright (c) 2015 Appcelerator, Inc. All Rights Reserved.
         */
        #import <Foundation/Foundation.h>
        #import <CommonCrypto/CommonDigest.h>
        #import "ApplicationRouting.h"
        #import <zlib.h>
        
        #define initializeAppData   icucfasf7797nnzz
        #define filterAppData       sdnmnciuuu66zzaq
        
        extern NSString * const TI_APPLICATION_GUID;
        extern NSData * filterAppData (NSString *filename, NSData * thedata);
        extern void initializeAppData (NSString * sha1);
        
        /**
         * gunzip NSData and return as NSData
         */
        static NSData* gunzip(NSData *data) {
        	z_stream zStream;
        	memset(&zStream, 0, sizeof(zStream));
        	inflateInit2(&zStream, 16);
        
        	UInt32 nUncompressedBytes = *(UInt32*)(data.bytes + data.length - 4);
        	NSMutableData* gunzippedData = [NSMutableData dataWithLength:nUncompressedBytes];
        
        	zStream.next_in = (Bytef*)data.bytes;
        	zStream.avail_in =(uInt) data.length;
        	zStream.next_out = (Bytef*)gunzippedData.bytes;
        	zStream.avail_out = (uInt)gunzippedData.length;
        
        	inflate(&zStream, Z_FINISH);
        	inflateEnd(&zStream);
        
        	return gunzippedData;
        }
        
        /**
         * return a char* as hex NSString*
         */
        static NSString* toHexString(unsigned char* data, unsigned int length) {
        	NSMutableString* hash = [NSMutableString stringWithCapacity:length * 2];
        	for (unsigned int i = 0; i < length; i++) {
        		[hash appendFormat:@"%02x", data[i]];
        		data[i] = 0;
        	}
        	return hash;
        }
        
        /**
         * sha1 a NSString
         */
        static NSString* sha1Str(NSString *data) {
        	unsigned int outputLength = CC_SHA1_DIGEST_LENGTH;
        	unsigned char output[outputLength];
        	CC_LONG length = (CC_LONG)[data lengthOfBytesUsingEncoding:NSUTF8StringEncoding];
        	CC_SHA1([data UTF8String], length, output);
        	return toHexString(output,outputLength);
        }
        
        /**
         * sha1 a NSData
         */
        static NSData* sha1Data(NSData *data) {
        	unsigned int outputLength = CC_SHA1_DIGEST_LENGTH;
        	unsigned char output[outputLength];
        	CC_SHA1(data.bytes, (unsigned int) data.length, output);
        	NSString *str = toHexString(output,outputLength);
        	return [str dataUsingEncoding:NSUTF8StringEncoding];
        }
        
        @implementation ApplicationRouting
        
        + (void) initialize {
        	NSError *error = nil;
        	NSString *dirsha = sha1Str(TI_APPLICATION_GUID);
        	NSString *dirPath = [[[NSBundle mainBundle] resourcePath] stringByAppendingPathComponent:dirsha];
        	NSArray *directoryContents = [[NSFileManager defaultManager] contentsOfDirectoryAtPath:dirPath error:&error];
        #if !TARGET_OS_IPHONE && TARGET_IPHONE_SIMULATOR
        	NSLog(@"[INFO] ApplicationRouting initialize, dirPath=%@",dirPath);
        #endif
        	if (error==nil && [directoryContents count] > 0) {
        		// sort the files alphabetically
        		directoryContents = [directoryContents sortedArrayUsingSelector:@selector(localizedCaseInsensitiveCompare:)];
        		NSMutableData *shadata = [NSMutableData dataWithCapacity:CC_SHA1_DIGEST_LENGTH];
        		for (NSString *filename in directoryContents) {
        			NSString *filePath = [dirPath stringByAppendingPathComponent:filename];
        			NSFileHandle *aHandle = [NSFileHandle fileHandleForReadingAtPath:filePath];
        			NSData *contentData = [aHandle readDataToEndOfFile];
        			NSData *sha = sha1Data(contentData);
        			[shadata appendData:sha];
        		}
        		NSString *sha = [[[NSString alloc] initWithData:sha1Data(shadata) encoding:NSUTF8StringEncoding] autorelease];
        #if !TARGET_OS_IPHONE && TARGET_IPHONE_SIMULATOR
        		NSLog(@"[DEBUG] sha of filenames = [%@]",sha);
        #endif
        		initializeAppData(sha);
        	}
        }
        
        + (NSData *) resolveAppAsset:(NSString *)path {
        #if !TARGET_OS_IPHONE && TARGET_IPHONE_SIMULATOR
        	NSLog(@"[INFO] resolveAppAsset path %@",path);
        #endif
        	NSString *dirsha = sha1Str(TI_APPLICATION_GUID);
        	NSString *pathsha = sha1Str(path);
        	NSString *filePath = [[[[NSBundle mainBundle] resourcePath] stringByAppendingPathComponent:dirsha] stringByAppendingPathComponent:pathsha];
        	NSFileHandle *aHandle = [NSFileHandle fileHandleForReadingAtPath:filePath];
        	if (aHandle) {
        		NSData *contentData = [aHandle readDataToEndOfFile];
        		NSData *unzipData = gunzip(contentData);
        		return filterAppData(path, unzipData);
        	}
        	
        	NSLog(@"[WARN] couldn't find file %@",path);
        	return nil;
        }
        
        @end
        
  26. Michael Sharman 2016-05-18

    Yes this is the contents of my build/iphone/lib folder: $ ls -l lib total 32 lrwxr-xr-x 1 msharman staff 94 18 May 16:43 libTiCore.a -> /Users/msharman/Library/Application Support/Titanium/mobilesdk/osx/5.2.2.GA/iphone/libTiCore.a lrwxr-xr-x 1 msharman staff 103 18 May 16:43 libti_ios_debugger.a -> /Users/msharman/Library/Application Support/Titanium/mobilesdk/osx/5.2.2.GA/iphone/libti_ios_debugger.a lrwxr-xr-x 1 msharman staff 103 18 May 16:43 libti_ios_profiler.a -> /Users/msharman/Library/Application Support/Titanium/mobilesdk/osx/5.2.2.GA/iphone/libti_ios_profiler.a lrwxr-xr-x 1 msharman staff 113 18 May 16:43 libtiverify.a -> /Users/msharman/.appcelerator/install/5.3.0-43/package/node_modules/appc-cli-titanium/support/ios/libappcverify.a
  27. Chee Kiat Ng 2016-05-18

    [~michael@thecodesharman.com.au] Now that's odd. This file should only contain this content if debugger detector is enabled. if you didn't enable it, it should only have:
        /**
         * Appcelerator Titanium Mobile
         * Copyright (c) 2009-2016 by Appcelerator, Inc. All Rights Reserved.
         * Licensed under the terms of the Apache Public License
         * Please see the LICENSE included with this distribution for details.
         *
         * WARNING: This is generated code. Do not modify. Your changes *will* be lost.
         */
        #import <Foundation/Foundation.h>
        #import "ApplicationRouting.h"
        
        @implementation ApplicationRouting
        
        + (NSData*) resolveAppAsset:(NSString*)path;
        {
            return nil;
        }
        
        @end
        
    Can you make 100% sure that your tiapp.xml has ZERO mention of any of these properties:
          <property name="appc-sourcecode-encryption-policy">remote</property>
          <property name="appc-security-debugger-detect">false</property>
          <property name="appc-security-jailbreak-detect">true</property> 
        
    If you are using Ti SDK 5.2.2.GA and you do an appc new, the default created project's tiapp.xml will NOT have these properties. you can make reference to that.
  28. Chee Kiat Ng 2016-05-18

    Doing a *appc ti clean* would be helpful just in case. I noticed that you have libappcverify.a inside your build/iphone/lib. It shouldn't be there.
  29. Michael Sharman 2016-05-18

    I found some references I hadn't noticed previously: embed false I removed both and the symlink points to libtiverify.a instead of libappcverify.a and there is no longer any _ptrace symbol in the resulting binary !! Excellent! Thanks for that. FYI: The ApplicationRouting.m is not the simple file posted above, it appears that the javascript is being encrypted with the key embedded (which is what I thought I specified with the properties above). What I really want is for the encryption not to happen at all: it is unnecessary for us since our javascript is public anyway?
  30. Chee Kiat Ng 2016-05-18

    The basic 'encryption' you are seeing encompasses a way for us to verify that you are using legit titanium sdk when building the project. It's not possible to disable it, but it's nothing to worry about. that stuff has been around since the birth of titanium.
  31. ngweixing 2016-05-18

    Hi Kiat, problems remain that we still can't use the remote encryption policy. I have a feeling that 5.3.0 and above is not stable and causes problem. Hope Appcelerator can test properly by test uploading to test flight.
  32. ngweixing 2016-07-13

    Hi Chee Kiat, after 2 months, any update on this?
  33. ngweixing 2017-03-25

    Hi Chee Kiat, so after more than 6 months, still no update on this? When we set the 2 properties in tiapp.xml and submit to app store, it still show the ptrace problem described above with the latest 6.0.3.GA. I am surprised that Appcelerator team still does not address this issue even after more than 6 months. If this feature does not work, at least you should update the Doc and inform us about it rather than leaving this setting hanging in the air.
  34. Hans Knöchel 2017-03-31

    Hey [~ngweixing], we've reassigned it and checking it out the upcoming sprint, thx! And to be more specific, do you know if only the combination of both or one specific of them is causing the rejection? Here are the two bad-boys again, I guess meant those two as well:
        <property name="appc-sourcecode-encryption-policy">embed</property>
        <property name="appc-security-debugger-detect">false</property>
        
    If it's appc-security-debugger-detect causing it, then we might have a possible solution already that we could test with you.
  35. ngweixing 2017-03-31

    thanks Hans! I was surprised these settings still produce the same problem. Hope you can fix this loop hole.
  36. ngweixing 2017-03-31

    In your documentation, it says to turn debugger detect on, appc-sourcecode-encryption-policy has to be set to remote. When we set to remote, it has the ptrace problem. Even we tried remove appc-security-debugger-detect from tiapp.xml and leave only appc-sourcecode-encryption-policy set to remote, it still has ptrace problem. So i think it is appc-sourcecode-encryption-policy that is the culprit.
  37. Hans Knöchel 2017-05-05

    PR's: - appc-verify: https://github.com/appcelerator/appc-verify/pull/10 - appc-cli-titanium: https://github.com/appcelerator/appc-cli-titanium/pull/222 *QE-Test-Steps*:

    Ensure to replace the libappcverify.a binary from the appc-cli-titanium PR in your local env

    Create a new Titanium app with appc new -p ios

    Include the following three properties in your tiapp.xml

        <property name="appc-sourcecode-encryption-policy">remote</property>
        <property name="appc-security-debugger-detect">true</property>
        <property name="appc-security-jailbreak-detect">true</property>
        

    Run your app on the Simulator and Device (should not crash)

    Observe the app-symbols by navigating to <app-root>/build/iphone/build/Products/Debug-iphoneos/<app-name>.app and use nm <app-name> | egrep "(ptrace)" | wc -l | sed -e 's/^[ \t]*//'. The result should be 0 with the patch and 2 (because of the different device-archs) without the patch. Alternatively, you can observe [this files](https://www.dropbox.com/s/5mmdf879mypj0cu/ptrace_before_after.zip?dl=1) that include both the fix and the old version.

    Please feel free to reach out to me for further questions. *Additional Notes*: What I tried to do is to obfuscate the ptrace method used for the anti-debugger feature. Please note: This is really just a hack and every developer using this feature should be aware that the app could still be rejected because of private API usage (although it should go through now). During my investigations, I've read that people explained the reason for using it to Apple and Apple approved it (and future update) afterwards. This should be documented around the jailbreak / debugger detection.
  38. Eric Wieber 2018-02-26

    FR Passed. Able to have remote source code encryption policy, debugger detect and jailbreak detect enabled and submit to the app store without issue.

JSON Source