[TIMOB-12866] Create c-based library to read source map
| GitHub Issue | n/a |
|---|---|
| Type | Sub-task |
| Priority | Low |
| Status | Closed |
| Resolution | Won't Do |
| Resolution Date | 2020-06-30T14:43:55.000+0000 |
| Affected Version/s | n/a |
| Fix Version/s | n/a |
| Components | Core |
| Labels | n/a |
| Reporter | Ingo Muschenetz |
| Assignee | Unknown |
| Created | 2013-02-26T19:45:46.000+0000 |
| Updated | 2020-06-30T14:43:55.000+0000 |
Description
The source map file described by https://github.com/mozilla/source-map needs to be parseable by our framework. If we can't use the JavaScript version of the library, we'll need to convert it to a language we can use across platforms, likely C.
See here for links to a Java-based version (Apache License) as part of the Google closure compiler.
I talked with Max through our source map use cases, and the pros and cons of various supporting architectures. We decided the best approach is to transcode the source map to a SQLite database at build time and deploy this database with the app. Our reasons for this decision are: 1) Mobile apps only require fast query of the source map. This is best handled by mobile databases (e.g. SQLite). 2) Source maps tend to be large (i.e. at least 3X the size of the original JavaScript), and mobile apps operate in an environment with limited resources. This is best handled by mobile databases (e.g. SQLite) which are designed to provide a small memory footprint by storing the indexes on disk. We do the memory intensive work of parsing and decoding the source map at build time and encoding it into an efficient query-able format. 3) SQLite is supported on all of our mobile platforms that require source map functionality. 4) We can reuse two existing JavaScript node modules to do the source map <-> SQLite transcoding, which is easily integrated into the CLI: 4.1) We'll use the node "source-map" module to parse and decode the source map (see https://github.com/mozilla/source-map). 4.2) We'll use the node "node-sqlite" module to encode the source map data into a SQLite database (see https://github.com/orlandov/node-sqlite). The task breakdown for this ticket is: 1. Design the SQLite schema for source map data. 2. Extend the CLI to transcode the source map data into a SQLite database. 3. Extend all titanium_mobile platforms to query this SQLite database when reporting errors to Crittercism. On iOS this is TiScriptError and TiExceptionHandler.
1. Reviewed source map specs. 2. Identified our specific on-device use cases. 3. Discussed with team the pros/cons of different architectures. 4. Created prototype node package for parsing/decoding source map data.
A few questions: 1. Is this an Enterprise-only feature? Or will it be open sourced? 2. When I deliver this feature after the 3.1 code freeze date, which repository does it belong in? This feature is a standalone utility that generates SQLite database files from source map files created by the CLI. This SQLite database file is a build artifact that will be an app Resource. Therefore, I'm thinking it should live in it's own github repository because it is standalone. Thoughts?
{noformat} ** Schema This is the SQLite3 schema for source maps. See below for more details about each table and column. +------------------+ | mappings | |------------------| | generated_line | +-----------+ | generated_column | | sources | | original_line | +---------+ |-----------| | original_column | | names | | source_id |<------+| source_id | |---------| | source | | name_id |+----->| name_id | +-----------+ +------------------+ | name | +---------+ ** Table: mappings The "mappings" table maps a {line, column} from the "generated" JavaScript source file (e.g. minified and/or obfuscated) to the corresponding {line, column} of the "original" JavaScript source file and possible a symbol name. |------------------+---------+---------------------------| | Column Name | Type | Comments | |------------------+---------+---------------------------| | generated_line | integer | primary key | | generated_column | integer | primary key | | original_line | integer | | | original_column | integer | | | source_id | integer | foreign key (never null) | | name_id | integer | foreign key (may be null) | |------------------+---------+---------------------------| ** Table: sources The "sources" table contains the original JavaScript source file names. |-------------+---------+----------------------| | Column Name | Type | Comments | |-------------+---------+----------------------| | source_id | integer | primary key | | source | text | original source file | |-------------+---------+----------------------| ** Table: names The "names" table contains the original JavaScript symbol names. |-------------+---------+---------------------------------| | Column Name | Type | Comments | |-------------+---------+---------------------------------| | name_id | integer | primary key | | name | text | original JavaScript symbol name | |-------------+---------+---------------------------------| {noformat}
{noformat} ** Example SQL SELECT #+begin_src sh #!/usr/bin/env bash
This bash script demonstrates querying a SQLite3 "source map"
database. It takes three arguments:
#1. generated_line
2. generated_column
3. path to the SQLite3 database file
#It then invokes the sqlite3 command line utility with these
arguments by parameterizing a SQL SELECT that returns information
about the original JavaScript source.
GENERATED_LINE=$1 GENERATED_COLUMN=$2 SQLITE3_DBFLE=$3 sqlite3 "${SQLITE3_DBFLE}" <Someone with privs to create repos under https://github.com/appcelerator needs to do it (I don't have privs). I suggest the repo name be "node-sm2sqlite".
Thank you Ingo. I have forked this repo, added the code, and submitted a pull request. Who should we assigned to it?
Max seems like a good candidate.
Made changes per Max and Chris's comments. All files now contain the Appcelerator licensing header and were refactored to follow Appcelerator's JavaScript coding standards. This is the PR: https://github.com/appcelerator/node-sm2sqlite/pull/2