[TIMOB-13983] Android: Titanium should use the latest stable Android SDK and NDK and not depend on 2.2 API 8
GitHub Issue | n/a |
Type | New Feature |
Priority | High |
Status | Closed |
Resolution | Fixed |
Resolution Date | 2013-10-19T01:23:34.000+0000 |
Affected Version/s | n/a |
Fix Version/s | 2013 Sprint 21, 2013 Sprint 21 Core, Release 3.2.0 |
Components | Android, Tooling |
Labels | androidbuild, cb-verified, module_android, qe-testadded |
Reporter | Eric Wing |
Assignee | Chris Barber |
Created | 2013-05-24T06:52:45.000+0000 |
Updated | 2014-04-18T01:14:26.000+0000 |
Description
Titanium should be using the latest stable Android SDK and NDK.
The Titanium environment seems to have some kind of implicit dependencies on building with the 2.2 SDK (API 8). The latest is 4.2.2 (API 17). I've been doing lots of fresh installs on Mac, Windows, and Linux recently. It seems that I am unable to build a simple Single Window template project without installing the 2.2 SDK. Also, using ant to build an Android Ti module, I am getting a complaint about 'No android-10' or 'android-2.3.3' in the Android SDK in the 'docgen' phase which I think is also coming from Titanium's build scripts.
"Best practices" for Android are to use/build the latest tools/SDK/NDK and set backwards compatibility targets to the lowest supported version you want to handle. This follows the Mac/iOS model which also expects you to build with the latest SDK and set minimum deployment versions independently.
While Android is quite terrible in their messaging about this, this is how things should be done. There are several key reasons that demonstrate why it needs to be done this way:
For module writers (like myself), we need the latest APIs available which is a problem since Titanium's build process is automatically invoking the wrong SDK on my behalf. I understand it is my responsibility to handle/test backwards compatibility, but you do me no favors by locking me to the oldest SDK. For example, OpenSL ES is the only sane path forward on trying to get low-latency audio on Android. It is only available starting in API 9 (2.3), and newer APIs aren't available until API 11 or 14. But if I write everything against 2.2, I cannot give any newer devices better audio. Furthermore, there were actually bugs on 2.3 devices where if you used the legacy APIs, things broke.
Android has dynamic launchtime/runtime behaviors based on how the apk's were compiled. In Android 4.0 for "Project Butter", GPU acceleration for native views is enabled by default, generally yielding better responsiveness for UI. However, if Android detects a legacy binary, it will avoid running this path of code and instead try to run with hardware acceleration off. By building against the 2.2 SDK, new projects look like legacy binaries to the system. I know from experience from the Apple/iOS side that legacy binaries on new versions of the OS are actually not the heavily tested path and I have first hand experienced very painful, hard-to-track down bugs that were solely introduced because of building against a legacy SDK and hitting special code paths that are not as well tested.
Downloading the Android SDK (ADT bundle) now always includes the latest SDK. Links to the older NDKs cannot be found from the main Android page. Users are not really expected to have to download old versions of either.
It is confusing for new users to know they have to download the 2.2 SDK (API 8). Why not the latest? Why not the oldest? Why not the market share dominate version (e.g. 2.3)?
Since the latest is installed when you first download, it's also a lot of wasted disk space to download older ones. I like to set up virtual machines for build environments (automation). So it adds up after awhile.
Ideally, you would look for the latest SDK installed on a user's system and use it, unless they provide some hard override.
Slightly less ideally, you require users to be on the latest SDK if you can't deal with dynamic detection. (Apple does this. We got used to it. It's actually easier in many respects because you get to say 'no' instead of test every permutation.)
Comments
JSON Source
From the blog: http://developer.appcelerator.com/blog/2013/05/3-1-1-release-candidate-of-titanium-sdkstudio-now-available.html {quote}3.1.1 now has a minimum supported Android version of 2.3.3 (API level 10).{quote} Resolving as fixed.
I do not think this is fixed because I've been using 3.1.1 builds to test for many weeks now and unless this was fixed over the weekend, I don't think this is really addressed nor do I think you fully understand what this bug is about. All you need to do to reproduce this problem is download a fresh SDK of Android. It should only contain 4.2.2 (API 17) SDK. Build the Single Window Application template project in Titanium Studio and the build will fail with stuff like:
The point is you are building against a legacy SDK, not the current one. There should be no mention of android-10 or android-8 or any other number except android-17 (as of this writing).
[~ewing] I agree that this shouldn't be closed, but I also suggest that always using the latest isn't wise either. We have a shifting sand of constantly updated SDKs that we rely on. Having at least one known good version means we know we can compile against that, though people should be free to use a newer version as well. Can't you update to build against a newer SDK if you so choose to do so?
No, I completely disagree :) The Android SDKs are not updating *that* frequently. Android API 17 (4.2) was released in November 2012. API 16 (4.1) was released in June 2012. API 14 (4.0) was released in October 2011. API 8 (2.2) was May 2010. 2.2 and 2.3 are several years old. There really is no excuse to be this far behind. I will concede that when Android does release a new SDK, you should have a grace period. But as a premiere platform and leader in the industry, you guys should be on top of this. I say you be ready to go within a month of release. And reiterating on my original points in the bug, when a new device is released, it is extremely risky to be using legacy binaries on brand new devices shipping later OS's because the manufacturers are not heavily testing this case. And from my first hand experience on the iOS side, you need to be ready to go the day Apple releases the new product because of this very problem iPad retina deployed with only a week announcement and triggered various display bugs if running a legacy binary. With Apple, you get beta periods so you can actually be ready the day they ship. Since Android doesn't necessarily do this, I think a month grace period is reasonable. And Ingo, to your question about me simply updating to a newer SDK, the answer is no, I can't do that. It doesn't work in Titanium. The above error output in my last comment is what I get when I try doing exactly that. There seems to be hardcoded stuff in Titanium that is looking for older SDKs.
Yeah I probably shouldn't have just resolved it so quickly because it is a wider issue than simply satisfying: {quote}Why not the market share dominate version (e.g. 2.3)?{quote} Like Ingo said it doesn't restrict you from building against the latest Android SDK or API level, the only issue here is that it's a bit of an inconvenience to have to install the API 10 SDK. The problem is that if the user had an SDK installed which is not entirely compatible or stable to be used with our platform (say, the very latest, or 3.x), that would introduce a whole new array of problems, especially for the users that are not technically versed enough to care about what SDK they're building against.
Lots of interesting ideas being floated. IMO we need to take a look at this in the 3.2 timeframe. QE will front load some investigation and will reply back with our findings.
Hello Eric, I think what Ingo was suggesting is this: 1) Open run configurations for your Studio project (right-click and "run as" then "run configurations") 2) In the run configuration screen, choose the Android SDK you wish to use. Or, when packaging, you are presented in the dialog with a choice of SDK.
I posted too late :) {quote}And Ingo, to your question about me simply updating to a newer SDK, the answer is no, I can't do that. There seems to be hardcoded stuff in Titanium that is looking for older SDKs.{quote} It looks for older SDKs, we do have a dependency on the API 10 for sure, and that's the topic here, but it doesn't stop you from using the latest SDK for your project. I can simply specify what API level I'd like to use in the manifest and it works for me.
That "Why not the market share dominate version (e.g. 2.3)?" was rhetorical. You shouldn't. I already explained it should be the latest stable. So I can't verify your claim that I can build with the latest SDK if I have android-10 installed. Why do I need this? This seems like a silly bug. How do I verify that the toolchain actually is built against the latest instead of accidentally pulling in android-10? As I said, Android has hidden runtime behaviors determining how things are built. I know for a fact that API 17 is used if I have no trace of anything older on my system. In Apple/Xcode, the SDK option lets you pick either: "Latest SDK" or a specific SDK version like "6.1" or "6.0". To handle the potential user confusion problem in Titanium, I think you should do something similar and set the default to "Latest SDK" and look for the most recent on their system. You could warn the user if you know that their latest is outdated. (You already have an updater that nags me way too often in my opinion for out-of-date Titanium components.) However, I think it is way more confusing that I am forced to install Android-8 or Android-10. This has already caused mass confusion in dealing with large teams that need reproducible build environments because when they go to Google and download the Android SDK/NDK, they automatically have incompatible/missing versions of Android, and nobody on the teams really know which other versions they need to install. (We've been guessing which SDK's we needed. I had it narrowed down to either android-8 or android-10 or both.) And already, I've gotten support issues from our downstream users that are trying to write their own modules for Titanium that need newer APIs and are hitting similar problems. You already have a confusion problem. It's just not the way you think it is.
Eric Merriman: Yes, I have set my Android SDK to my 4.2.2 in preferences and also how you described. I get the failure messages pasted earlier. (This is with a default Android SDK installation of only 4.2.2, so there is no android-10 or android-8 installed. I know installing both will get me through the build errors and compile the app, but I no longer know which SDK was actually "used" despite the GUI setting. I shouldn't need those legacy ones if I'm truly using my specified option.)
While there's no clear visual indicator for which SDK it's using, you can be sure that it's using whatever you specify in the "tool-api-level" in the manifest:
If you do a quick search of the build log for "/android.jar" you can see which SDK version it's pulling in. In my build log:
If I specify a tool level of 10:
Hello, A few points of clarity: 1) We strive to support the SDKs (always including latest) that have the highest installed market share. As of last check, we dropped 2.2 from our supported list. From a QE standpoint we generally don't want to continue to maintain many supported SDKs, since the effort multiplies our matrix. But since Enterprise wants to target a broad range of OS versions, we maintain this support. 2) Android has broken us many times. This is why we jump on the latest tools as quickly as possible. Remember, in addition to the SDKs, there are the tools, and the other components. We will continue to strive to support the latest versions as soon as they are available. 3) The list provided of Android SDK releases contains 4 items, but is missing a few: API 9 2.3 2.3.1 2.3.2 API 10 2.3.3 2.3.4 2.3.5 2.3.6 2.3.7 API 11 3.0 API 12 3.1 API 13 3.2 API 15 4.0.3 4.0.4 4) The checks you see in the console were originally performed to ensure that the user had a valid Android setup. The plan was to update the legacy Android python scripts to the newer paradigm that is smarter about detecting a valid environment. When this happens it will be easier to allow more flexibility in the required environment. 5) It is difficult to not be frustrated with this ticket when you will not install our required Android SDKs (for studio to build) and so will not do anything to validate your original claim that resulting build will contain non 4.2.2 components.
Focusing on what's in Titanium today, if I install both the default API 17 and also API 10 and do what Eric Merriman suggests by setting my Target SDK to 4.2.2/17, it looks like it does not do what I want and builds against android-10 SDK. [INFO] logfile = /Volumes/DataPartition/Users/ewing/Documents/Titanium_Studio_Workspace/SingleWindowTemplate2/build.log [DEBUG] /Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/builder.py install SingleWindowTemplate2 /Library/Frameworks/Android/android-sdk [DEBUG] /Library/Frameworks/Android/android-sdk/platform-tools/aapt package -m -J /Volumes/DataPartition/Users/ewing/Documents/Titanium_Studio_Workspace/SingleWindowTemplate2/build/android/gen -M /Volumes/DataPartition/Users/ewing/Documents/Titanium_Studio_Workspace/SingleWindowTemplate2/build/android/AndroidManifest.xml -S /Volumes/DataPartition/Users/ewing/Documents/Titanium_Studio_Workspace/SingleWindowTemplate2/build/android/res -I /Library/Frameworks/Android/android-sdk/platforms/android-10/android.jar [DEBUG] javac -encoding utf8 -classpath "/Library/Frameworks/Android/android-sdk/platforms/android-10/android.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/modules/titanium-media.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/jaxen-1.1.1.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/kroll-common.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/ti-commons-codec-1.3.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/android-support-v4.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/modules/titanium-platform.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/titanium.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/modules/titanium-app.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/modules/titanium-analytics.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/kroll-v8.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/modules/titanium-android.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/modules/titanium-ui.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/modules/titanium-locale.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/modules/titanium-filesystem.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/lib/titanium-verify.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/lib/titanium-debug.jar:/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/lib/titanium-profiler.jar" -d /Volumes/DataPartition/Users/ewing/Documents/Titanium_Studio_Workspace/SingleWindowTemplate2/build/android/bin/classes -proc:none -sourcepath /Volumes/DataPartition/Users/ewing/Documents/Titanium_Studio_Workspace/SingleWindowTemplate2/build/android/src -sourcepath /Volumes/DataPartition/Users/ewing/Documents/Titanium_Studio_Workspace/SingleWindowTemplate2/build/android/gen -target 1.6 -source 1.6 @/var/folders/28/2sx8t8s535vfhcbjl7fb3x680000gp/T/tmpf3FpTg [DEBUG] /Library/Frameworks/Android/android-sdk/platform-tools/aapt package -f -M AndroidManifest.xml -A /Volumes/DataPartition/Users/ewing/Documents/Titanium_Studio_Workspace/SingleWindowTemplate2/build/android/bin/assets -S res -I /Library/Frameworks/Android/android-sdk/platforms/android-10/android.jar -I "/Volumes/DataPartition/Users/ewing/Library/Application Support/Titanium/mobilesdk/osx/3.1.1.v20130529114554/android/titanium.jar" -F /Volumes/DataPartition/Users/ewing/Documents/Titanium_Studio_Workspace/SingleWindowTemplate2/build/android/bin/app.ap_ However, if I add the lines to the tiapp.xml:
In response to the API SDK versions, the SDK releases are generally tied to API version changes and not necessarily point releases. I didn't list all the point versions because I don't think they are the relevant thing to focus on. To the main point though, I'm arguing that building against legacy SDKs is not correct. (And it was by no means clear/obvious how to get around this in Titanium.) It is a bad and incorrect assumption that building with a legacy SDK some how "locks you into" existing behavior. In my experience, this is not what actually happens and instead you are now hitting runtime compatibility fallback code paths that are not well tested on newer OS versions and newer devices that ship with these OS versions. Depending on locked-down legacy versions is a false form of security in this realm. (This works better in the Windows realm.) I agree supporting Android is very painful and share your pain. However, users don't care about this and all that is important is they get the best user experience possible. In most cases, this means relying on the latest features, behaviors, and bug fixes in the latest OS's and SDKs (like Project Butter) and developers dealing with the pain of writing fallbacks and testing for the older devices. And that is why we are here. An addition point to understand is that the mobile market is still growing rapidly. There are two important metrics to remember. 1) The expected life span of a device is only 2 years (due to phone contract renewal cycles) 2) Every year, more new devices are sold than all previous years combined. This means there is an experience bias towards the latest even if nobody gets to upgrade in Android. (Android 4.0 + 4.1 + 4.2 do out weigh 2.3 and doesn't even account for filters about whether older device users actively download new apps or upgrades.)
{quote}Where does "tool-api-level" come from? I have never seen this in the Android docs. I have seen targetSdkVersion (and have an open question from a downstream user about why it doesn't seem to work in Titanium){quote} It's the version of the Android API that Titanium builds against, as you requested. {quote}Why is
I added the "targetSdkVersion" as well; the uses-sdk values are used to inform the Android platform of your app's compatibility support (not related to tool-api-level, but still important to include if you're targeting OS-specific features and themes). See: http://developer.android.com/guide/topics/manifest/uses-sdk-element.html So really most of this is just a misunderstanding, it's all in the docs. The real issue here is the point that Eric M made: {quote}The plan was to update the legacy Android python scripts to the newer paradigm that is smarter about detecting a valid environment.{quote} That's a fair request, and we can look into that. Would you be satisfied with that?
Yes, this is all terrific! (I hope I'm not sounding like a complete ass in this bug report. I appreciate all the discussion here. My goals are mid-term oriented to make sure Titanium is building optimal binaries, allow us to access to the modern/bleeding-edge features we need, and simplify the setup and build environments for the teams we are working with, automation/continuous integration environments we need to make, and downstream developers that are working with both our products. (If you don't know, we are a game module for Titanium)).
This is all fixed in the new Android build. We have proper target SDK support and you no longer need Android SDK 2.2 or 2.3 just to build with 4.3 (or whatever). The NDK is not required for building apps, only modules, so that's a non-issue.
PR: https://github.com/appcelerator/titanium_mobile/pull/4781
To test: Remove all Android SDKs except your target. Try from Studio (may not detect SDK) and the CLI.
Verified deleting all the android SDK's except android 4.4 & built for device from studio & CLI. Environment: Appcel Studio : 3.2.0.201312042306 Ti SDK : 3.2.0.v20131204220843 Mac OSX : 10.8.5 Alloy : 1.3.0-beta CLI - 3.2.0-beta Samsung Galaxy S4 running android 4.2.2 Nexus 5 - android 4.4
This still breaks for modules on 3.2.2.GA. Was being tracked under TISTUD-5558 but that is closed, as it was against 3.2.0.v20131018154951.
[~athorne] The problem is that once you create a module, (generally) the build logic never gets updated unless you A) create a new module and copy your code over or B) manually hack it to be more recent. Android's Ant file (build.xml) will reference a parent build.xml inside the Titanium SDK (via the build.properites "titanium.platform" setting), but that doesn't get updated frequently with fixes. As of the latest Titanium SDK, it relies on Android SDK 2.3.3. So you must have that installed. I'm not sure what the required/supported version of the NDK is. FWIW, in Titanium SDK 3.3.0, it will use the oldest Android SDK that's at least 2.3.3.
Got it. Thank you, Chris. 3.3.0 sounds like a huge step forward in a lot of ways. Have a good one.