{ "id": "169031", "key": "TIMOB-24986", "fields": { "issuetype": { "id": "5", "description": "The sub-task of the issue", "name": "Sub-task", "subtask": true }, "parent": { "id": "168373", "key": "TIMOB-24816", "fields": { "summary": "Update the mocha suite in titanium_mobile_windows", "status": { "description": "The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.", "name": "Closed", "id": "6", "statusCategory": { "id": 3, "key": "done", "colorName": "green", "name": "Done" } }, "priority": { "name": "None", "id": "6" }, "issuetype": { "id": "6", "description": "gh.issue.epic.desc", "name": "Epic", "subtask": false } } }, "project": { "id": "10153", "key": "TIMOB", "name": "Titanium SDK/CLI", "projectCategory": { "id": "10100", "description": "Titanium and related SDKs used in application development", "name": "Client" } }, "fixVersions": [], "resolution": { "id": "10000", "description": "", "name": "Done" }, "resolutiondate": "2017-07-25T12:42:26.000+0000", "created": "2017-07-20T12:45:38.000+0000", "priority": { "name": "None", "id": "6" }, "labels": [], "versions": [], "issuelinks": [], "assignee": { "name": "eharris", "key": "eharris", "displayName": "Ewan Harris", "active": true, "timeZone": "Europe/Dublin" }, "updated": "2017-07-25T12:42:31.000+0000", "status": { "description": "The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.", "name": "Closed", "id": "6", "statusCategory": { "id": 3, "key": "done", "colorName": "green", "name": "Done" } }, "components": [], "description": "There are around 500 skipped tests on Windows, the total number of tests is only around 1000 so half of them being skipped is not good and they almost definitely do not have ticket associated with them if they're skipped for bugs, I know I have skipped some of these in frustration when merging the suites.\r\n\r\nReview the attached skipped list and then run through the tests, reviewing what can be unskipped as it now works, filing tickets as needed.\r\n\r\nI don't think we'd be able to achieve a code coverage output so in my opinion we should have a mechanism where we require a percentage of tests to be run to allow us to measure this, while it's not a great metric it is something to aim for and stop the bulk skipping of tests", "attachment": [], "flagged": false, "summary": "Investigate the skipped tests on Windows", "creator": { "name": "eharris", "key": "eharris", "displayName": "Ewan Harris", "active": true, "timeZone": "Europe/Dublin" }, "subtasks": [], "reporter": { "name": "eharris", "key": "eharris", "displayName": "Ewan Harris", "active": true, "timeZone": "Europe/Dublin" }, "environment": null, "closedSprints": [ { "id": 925, "state": "closed", "name": "2017 Sprint 15 SDK", "startDate": "2017-07-16T14:40:47.191Z", "endDate": "2017-07-30T14:40:00.000Z", "completeDate": "2017-07-31T16:15:09.335Z", "originBoardId": 114 } ], "comment": { "comments": [ { "id": "424452", "author": { "name": "eharris", "key": "eharris", "displayName": "Ewan Harris", "active": true, "timeZone": "Europe/Dublin" }, "body": "Ran through all but Ti.UI, here are my notes\r\n\r\n\r\n{code}\r\nNote that seems to run fine means the app doesn't crash, it does not mean the tests all pass :)\r\n\r\nTitanium.Analytics has 1 skipped tests\r\n\t- filterEvents() is not implemented on Windows\r\nTitanium.API has 1 skipped tests\r\n\t- timestamp() is not implemented on Windows and Android\r\nTitanium.App has 5 skipped tests\r\n\t- disableNetworkActivityIndicator is an iOS only thing\r\n\t- getDisableNetworkActivityIndicator is an iOS only thing\r\n\t- forceSplashAsSnapshot is an iOS only thing\r\n\t- getForceSplashAsSnapshot is an iOS only thing\r\n\t- searchQuery is an iOS only thing (also should be moved to an Ti.App.iOS)\r\nTitanium.App.Windows.BackgroundService has 6 skipped tests\r\n\t- Meh\r\nTitanium.Blob has 3 skipped tests\r\n \t- apiName unskipped\r\n \t- constructed from File.read() - It fails for some reason, unskipped though as it doesn't kill the app\r\n\t- constructed from image, accessing props on the blob seems to just kill the app :)\r\nGlobal has 1 skipped tests\r\n\t- global no idea why\r\nTitanium.Contacts has 27 skipped tests\r\n\t- Was skipped in frustration\r\n\t- should(queriedPerson).not.be.null; kills the app so skipped person add/remove. Other than that, all good in the hood\r\nTitanium.Contacts.Person has 32 skipped tests\r\n \t- Was skipped in frustration, unskipped, looks good?\r\nTitanium.Filesystem has 20 skipped tests\r\n\t- Suite was skipped, unskipped and runs fine with failures\r\nTitanium.Filesystem.File has 41 skipped tests\r\n\t- Suite was skipped, unskipped ad seems to run fine with failures, need to revisit when reduced\r\nTitanium.Filesystem.FileStream has 7 skipped tests\r\n\t- Suite was skipped, unskipped, runs fine with failures, but there are logs after the tests have finished?\r\nTitanium.Geolocation has 18 skipped tests\r\n\t- Prompting on mobile device family I think breaks the test run, we Sneed to decide how to handle these situations\r\nTitanium.Locale has 3 skipped tests\r\n\t- No crashes, just errors, unskipped them all\r\nTitanium.Map has 30 skipped tests\r\n\t- Seems to run fine, reminds me we need to split this out from core\r\nTitanium.Media.AudioPlayer has 10 skipped tests\r\n\t- Seems to run fine, unskipped!\r\nTitanium.Media.Sound has 10 skipped tests\r\n\t- Seems to run fine, unskipped!\r\nTitanium.Network.HTTPClient has 8 skipped tests\r\n\t- Network tests are inherently flakey due to environmental issues, I propose we remove these from unit tests or make it not dependent on an external network connection\r\nTitanium.Platform has 28 skipped tests\r\n\t- Looks to run fine, unskipped!\r\nTitanium.Platform.DisplayCaps has 14 skipped tests\r\n\t- Looks to run fine, unskipped!\r\nTitanium has 18 skipped tests\r\n\t- Looks to run fine, unskipped!\r\nTitanium.UI.Button has 1 skipped tests\r\n\t- image(Blob) crashes the app, other than that runs fine\r\nTitanium.UI has 147 skipped tests\r\n\t- All constants :/, might be a pita\r\nTitanium.UI.ImageView has 14 skipped tests\r\n\t- Just in a bad place right now, keeping skipped\r\nTitanium.UI.Label has 1 skipped tests\r\n\t- Runs fine, unskipped\r\nTitanium.UI.Layout has 16 skipped tests\r\n\t- Some tests in this suite (as stated in comments) seem kinda funky\r\n\t- TIMOB-23372 #10 crashes the app, skipped other than that runs fine\r\nTitanium.UI.ListView has 3 skipped tests\r\n\t- ListItem.properties is iOS only, other than that skipped\r\nTitanium.UI.ScrollView has 5 skipped tests\r\n\t- scrollType is android only\r\n\t- maxZoomScale is iOS only\r\n\t- minZoomScale is ios only \r\n\t- overScrollMode is android only\r\nTitanium.UI.Switch has 1 skipped tests\r\n\t- apiname unskipped\r\nTitanium.UI.TableView has 1 skipped tests\r\n\t- unskipped runs fine\r\nTitanium.UI.TextField has 2 skipped tests\r\n\t- unskipped runs fine\r\nTitanium.UI.WebView has 1 skipped tests\r\n\t- unskipped runs fine\r\nTitanium.UI.Window has 2 skipped tests\r\n\t- unskipped, runs fine\r\nTitanium.Utils has 1 skipped tests\r\n\t- unskipped, runs fine\r\nTitanium.XML has 12 skipped tests\r\n\t- unskipped, runs fine\r\n\r\n{code}", "updateAuthor": { "name": "eharris", "key": "eharris", "displayName": "Ewan Harris", "active": true, "timeZone": "Europe/Dublin" }, "created": "2017-07-20T15:56:59.000+0000", "updated": "2017-07-20T15:56:59.000+0000" }, { "id": "424534", "author": { "name": "eharris", "key": "eharris", "displayName": "Ewan Harris", "active": true, "timeZone": "Europe/Dublin" }, "body": "Current list of skipped tests that are need to be investigated, this means that either they crash the app, are noted as questionable tests in the code or are flaky (HTTPClient)\r\n\r\n{code}\r\nTitanium.App.Windows.BackgroundService has 6 skipped tests\r\n\tAPI\r\n\tregisterTimerTask\r\n\tregisterPushNotificationTask\r\n\tunregisterTask(task)\r\n\tunregisterTask(task id)\r\n\tunregisterAllTasks\r\nGlobal has 1 skipped tests\r\n\tglobal\r\nTitanium.Network.HTTPClient has 18 skipped tests\r\n\tapiName\r\n\tresponseXML\r\n\tsecureValidateProperty\r\n\tdownloadLargeFile\r\n\tTIMOB-23127\r\n\tTIMOB-23214\r\n\tTIMOB-19042\r\n\tlargeFileWithRedirect\r\n\temptyPOSTSend\r\n\tresponseHeadersBug\r\n\trequestHeaderMethods\r\n\tsendData\r\n\tclearCookiePositiveTest\r\n\tclearCookieUnaffectedCheck\r\n\tsetCookieClearCookieWithMultipleHTTPClients\r\n\tcallbackTestForGETMethod\r\n\tcallbackTestForPOSTMethod\r\n\tPOST multipart/form-data containing Ti.Blob\r\nTitanium has 1 skipped tests\r\n\tcreateProxy()\r\nTitanium.UI.ImageView has 2 skipped tests\r\n\timage (File)\r\n\timage (Blob)\r\nTitanium.UI.Layout has 1 skipped tests\r\n\tTIMOB-23372 #10\r\n{code}", "updateAuthor": { "name": "eharris", "key": "eharris", "displayName": "Ewan Harris", "active": true, "timeZone": "Europe/Dublin" }, "created": "2017-07-21T17:17:14.000+0000", "updated": "2017-07-21T17:17:14.000+0000" }, { "id": "424642", "author": { "name": "eharris", "key": "eharris", "displayName": "Ewan Harris", "active": true, "timeZone": "Europe/Dublin" }, "body": "Considering this done", "updateAuthor": { "name": "eharris", "key": "eharris", "displayName": "Ewan Harris", "active": true, "timeZone": "Europe/Dublin" }, "created": "2017-07-25T12:42:26.000+0000", "updated": "2017-07-25T12:42:26.000+0000" } ], "maxResults": 4, "total": 4, "startAt": 0 } } }