[TIMOB-19666] iOS9.1: Support PHLivePhoto + PHLivePhotoView

GitHub Issue	n/a
Type	New Feature
Priority	Medium
Status	Closed
Resolution	Fixed
Resolution Date	2016-02-16T05:45:53.000+0000
Affected Version/s	n/a
Fix Version/s	Release 5.2.0
Components	iOS
Labels	iOS9.1, notable, photo, photogallery, qe-5.2.0
Reporter	Chee Kiat Ng
Assignee	Hans Knöchel
Created	2015-10-05T06:17:25.000+0000
Updated	2016-02-17T18:53:39.000+0000

Description

*Summary:* iOS 9.1 provides a way to receive, display and share live photos using the PHLivePhoto related classes. A live photo is a photo, that consists of several photos taken together as frames but it still is no video. It can be displayed using the PHLivePhotoView and managed through the PHImageManager or UIImagePickerController (which we use). *Proposal:* Summarized, our proposal should be: * a) to support taking live photos using Ti.Media.openPhotoGallery() method * b) allow developers to display live photos in a new view proxy e.g. Ti.UI.iOS.LivePhoto or directly in the existing Ti.UI.ImageView using auto detection on the image source type. *Documentation:* * https://developer.apple.com/library/prerelease/ios/documentation/PhotosUI/Reference/PHLivePhotoView_Class/index.html#//apple_ref/doc/uid/TP40016582 * https://developer.apple.com/library/prerelease/ios/documentation/Photos/Reference/PHLivePhoto_Class/index.html#//apple_ref/doc/uid/TP40016581

Comments

Hans Knöchel 2015-11-25

PR pending: https://github.com/appcelerator/titanium_mobile/pull/7479 Demo (live photo view):

   var window = Ti.UI.createWindow({
     title: "iOS 9.1 Live Photos",
     backgroundColor: "#fff"
   });
   
   var view = null;
   
   var nav = Ti.UI.iOS.createNavigationWindow({
     window: window
   });
   
   var btn1 = Ti.UI.createButton({
     systemButton: Ti.UI.iPhone.SystemButton.ADD,
     top:30
   });
   
   btn1.addEventListener("click", openGallery);
   
   var btn2 = Ti.UI.createButton({
     systemButton: Ti.UI.iPhone.SystemButton.TRASH,
     top:10
   });
   
   btn2.addEventListener("click", resetImageView);
   
   window.setRightNavButton(btn1);
   window.setLeftNavButton(btn2);
   nav.open();
   
   function openGallery() {
     Ti.Media.openPhotoGallery({
       mediaTypes: [Ti.Media.MEDIA_TYPE_PHOTO, Ti.Media.MEDIA_TYPE_LIVEPHOTO],
       success: function(e) {
         var photo = e.media; // Static photo without motion of type Ti.Blob
         var livePhoto = e.livePhoto // Live photo of type Ti.UI.iOS.LivePhoto
   
         Ti.API.warn(e.mediaType); // Should display "com.apple.live-photo" when a live photo is selected
   
         if(livePhoto) {
           // Live photo supported and returned
           view = Ti.UI.iOS.createLivePhotoView({
             livePhoto: livePhoto,
             backgroundColor: "#f00"
           });
   
           view.addEventListener("start", function(e) {
               Ti.API.warn("Start playback");
               Ti.API.warn(e);
           });
   
           view.addEventListener("stop", function(e) {
               Ti.API.warn("Stop playback");
               Ti.API.warn(e);
           });
   
       } else if (photo && e.mediaType == Ti.Media.MEDIA_TYPE_PHOTO) {
           // Default photo for iOS < 9.1
           view = Ti.UI.createImageView({
             image: photo
           });
         } else {
           Ti.API.warn("Fallback: Most probably selected video (not part of this example).");
           return;
         }
         window.add(view);
       }
     });
   }
   
   function resetImageView() {
       var children = window.children;
   
       for(var i = 0; i < children.length; i++) {
           var child = children[i];
           window.remove(child);
       }
   }

Demo (live photo badges):

   var window = Ti.UI.createWindow({
     backgroundColor: "#fff",
     layout: "vertical"
   });
   
   addViewWithOptions("No options (default: Over content)");
   addViewWithOptions("Over content", Ti.UI.iOS.LIVEPHOTO_BADGE_OPTIONS_OVER_CONTENT);
   addViewWithOptions("Live off", Ti.UI.iOS.LIVEPHOTO_BADGE_OPTIONS_LIVE_OFF);
   addViewWithOptions("Bit-wise options", Ti.UI.iOS.LIVEPHOTO_BADGE_OPTIONS_OVER_CONTENT | Ti.UI.iOS.LIVEPHOTO_BADGE_OPTIONS_LIVE_OFF);
   
   function addViewWithOptions(title, options) {
   	var liveBadge = Ti.UI.iOS.createLivePhotoBadge(options);
   
   	var view = Ti.UI.createView({
   		width: 300,
   		height: 100,
   		top: 40,
   		backgroundColor: "#333"
   	});
   
   	var label = Ti.UI.createLabel({
   		color: "#fff",
   		text: title
   	});
   
   	var img = Ti.UI.createImageView({
   		image: liveBadge,
   		top: 10,
   		right: 10
   	});
   
   	view.add(label);
   	view.add(img);
   	window.add(view);
   }
   
   window.open();

Chee Kiat Ng 2015-12-01

CR and FT passed. PR merged.
Wilson Luu 2016-02-01

[~hansknoechel], [~cng], It looks like the LivePhoto is not working with Ti.Media.showCamera(). Should we reopen this ticket or file a new ticket? [~jlongton], please go ahead and paste your code that exhibit this bug.

Josh Longton 2016-02-01

   var window = Ti.UI.createWindow({
     title: "iOS 9.1 Live Photos",
     backgroundColor: "#fff"
   });
    
   var view = null;
    
   var nav = Ti.UI.iOS.createNavigationWindow({
     window: window
   });
    
   var btn1 = Ti.UI.createButton({
     systemButton: Ti.UI.iPhone.SystemButton.ADD,
     top:30
   });
    
   btn1.addEventListener("click", openCamera);
   // btn1.addEventListener("click", openGallery);
    
   var btn2 = Ti.UI.createButton({
     systemButton: Ti.UI.iPhone.SystemButton.TRASH,
     top:10
   });
    
   btn2.addEventListener("click", resetImageView);
    
   window.setRightNavButton(btn1);
   window.setLeftNavButton(btn2);
   nav.open();
   function openCamera(){
   	Ti.Media.showCamera({
   		mediaTypes: [Ti.Media.MEDIA_TYPE_PHOTO, Ti.Media.MEDIA_TYPE_LIVEPHOTO, Ti.Media.MEDIA_TYPE_VIDEO],
   		success: function(e) {
         		var photo = e.media; // Static photo without motion of type Ti.Blob
         		var livePhoto = e.livePhoto; // Live photo of type Ti.UI.iOS.LivePhoto
         		      if(livePhoto) {
   			        // Live photo supported and returned
   			        view = Ti.UI.iOS.createLivePhotoView({
   			          livePhoto: livePhoto,
   			          backgroundColor: "#f00"
   			        });
   			 
   			        view.addEventListener("start", function(e) {
   			            Ti.API.warn("Start playback");
   			            Ti.API.warn(e);
   			        });
   			 
   			        view.addEventListener("stop", function(e) {
   			            Ti.API.warn("Stop playback");
   			            Ti.API.warn(e);
   			        });
   			 
   			    } else if (photo && e.mediaType == Ti.Media.MEDIA_TYPE_PHOTO) {
   			        // Default photo for iOS < 9.1
   			        view = Ti.UI.createImageView({
   			          image: photo
   			        });
   			      } else {
   			        Ti.API.warn("Fallback: Most probably selected video (not part of this example).");
   			        return;
   			      }
         		window.add(view);
        	}
       });
   }
    
   // function openGallery() {
     // Ti.Media.openPhotoGallery({
       // mediaTypes: [Ti.Media.MEDIA_TYPE_PHOTO, Ti.Media.MEDIA_TYPE_LIVEPHOTO],
       // success: function(e) {
         // var photo = e.media; // Static photo without motion of type Ti.Blob
         // var livePhoto = e.livePhoto; // Live photo of type Ti.UI.iOS.LivePhoto
   //  
         // Ti.API.warn(e.mediaType); // Should display "com.apple.live-photo" when a live photo is selected
   //  
         // if(livePhoto) {
           // // Live photo supported and returned
           // view = Ti.UI.iOS.createLivePhotoView({
             // livePhoto: livePhoto,
             // backgroundColor: "#f00"
           // });
   //  
           // view.addEventListener("start", function(e) {
               // Ti.API.warn("Start playback");
               // Ti.API.warn(e);
           // });
   //  
           // view.addEventListener("stop", function(e) {
               // Ti.API.warn("Stop playback");
               // Ti.API.warn(e);
           // });
   //  
       // } else if (photo && e.mediaType == Ti.Media.MEDIA_TYPE_PHOTO) {
           // // Default photo for iOS < 9.1
           // view = Ti.UI.createImageView({
             // image: photo
           // });
         // } else {
           // Ti.API.warn("Fallback: Most probably selected video (not part of this example).");
           // return;
         // }
         // window.add(view);
       // }
     // });
   // }
    
   function resetImageView() {
       var children = window.children;
    
       for(var i = 0; i < children.length; i++) {
           var child = children[i];
           window.remove(child);
       }
   }

Hans Knöchel 2016-02-01

[~wluu] It was an issue in my proposal which I just corrected. For now (iOS 9.1), it's not possible to capture a new live photo but only to select an existing one. The docs should state that behavior as well. As soon as Apple makes it possible to capture live photos as well, I would file a new ticket and implement it as well. Thanks!
Wilson Luu 2016-02-01

[~hansknoechel], Okay, I'll file a doc ticket for this issue.

Fokke Zandbergen 2016-02-12

Reopening because with latest nightly (5.2.0.v20160211193617) I get LIVEPHOTO mediaType but no livePhoto:

     Ti.Media.openPhotoGallery({
       mediaTypes: [Ti.Media.MEDIA_TYPE_PHOTO, Ti.Media.MEDIA_TYPE_LIVEPHOTO],
       success: function(e) {
         Ti.API.warn(e.mediaType);
         Ti.API.warn(e.media);
         Ti.API.warn(e.livePhoto);
       }
     });

   [WARN]  com.apple.live-photo
   [WARN]  [object TiBlob]
   [WARN]  <null>

Hans Knöchel 2016-02-12

This issue happens when we want to receive a LivePhoto but no Ti.UI.LivePhotoView is defined. Although it's an edge-case, because live photos can only used be displayed inside a view, we should fix it to prevent questions. PR (master): https://github.com/appcelerator/titanium_mobile/pull/7717 PR (5_2_X): https://github.com/appcelerator/titanium_mobile/pull/7718
Hans Knöchel 2016-02-13

Considering to close the PR, since the issue is not caused by the implementation itself, but an issue when running on main thread (TIMOB-20397). [~fokkezb] as you re-opened it, what do you think?
Fokke Zandbergen 2016-02-13

Also, http://docs.appcelerator.com/platform/latest/#!/api/Titanium.UI.iOS.LivePhotoView-method-startWithPlaybackStyle does not work and can't find the implementation as wel: https://github.com/appcelerator/titanium_mobile/search?q=startWithPlaybackStyle
Hans Knöchel 2016-02-13

Fixed docs issue
Fokke Zandbergen 2016-02-14

[~hansknoechel] did you test the manual start and stop? I see it triggering the start and stop events but nothing happens for a few seconds after start and then it shows only a very short part of the animation. With hint even shorter. Also, when I call stop during this it kind of freezes. It doesn't feel like it works as it should. On the PR for using Ti.Media.MEDIA_TYPE_LIVEPHOTO; I still think we need to fix this. I don't see why it's related to TIMOB-20397 as it's simply a matter of the LivePhoto proxy API not being included in the build if you don't also use the view.
Hans Knöchel 2016-02-14

Regarding the start/stop: I was also ensure, but it looks like the native behavior, since it's not really a video so the sequence is really short (faded). The PR is still valid for making the Ti.UI.iOS.LivePhoto available, but the threading issues also fixed in the PR are unrelated so I would clean up the PR to only remove the #ifdef's and fix the main-thread-related issues i the separate ticket.
Fokke Zandbergen 2016-02-14

I've updated the 5.2.0 sample so you can see what I see. I doubt this is how it should look. You'd expect full to show the full length of the live photo and both of them not to have such a long delay before they do anything.
Hans Knöchel 2016-02-14

Got it and finally reproduced it. The methods had threading problems, I added the 1-liner to the existing PR. Also added a start/stop test to the example: https://gist.github.com/hansemannn/68252eb5c106c2552d21
Chee Kiat Ng 2016-02-15

Can we have a simple test case here for the new fix? QE will appreciate it. [~hansknoechel] [~fokkezb]
Hans Knöchel 2016-02-15

See Gist above :-)
Fokke Zandbergen 2016-02-15

[~hansknoechel] patched my SDK with he main thread fix and it works fine now.
Chee Kiat Ng 2016-02-16

CR and FT passed. PRs merged.
Josh Longton 2016-02-17

JSON Source