Problem description
Trying to use base64encode with a very large file, fails with an 'out of memory' error.
Use case
The use case would be to upload a base64-encoded file using an HTTPClient. The conversion to base64 fails.
Sample code
var filename = "SOME_LARGE_FILE";
var file = Titanium.Filesystem.getFile(filename);
var toUpload = file.read();
var ws = {
file_name: filename,
data: Ti.Utils.base64encode(toUpload)
};
var xhr = Ti.Network.createHTTPClient({
onload: function(e) {
Ti.API.info(this.responseText);
}
});
xhr.open('POST', 'SOME_UPLOAD_URL');
xhr.send(ws);
Modified the code to use the stream directly in the HTTPClient to avoid memory allocation. TiBlob is modified to support stream and take file as input. https://github.com/appcelerator/titanium_mobile/pull/5261
Modified the code to use the stream directly in the HTTPClient to avoid memory allocation. TiBlob is modified to support stream and take file as input. https://github.com/appcelerator/titanium_mobile/pull/5261
Run the following code to reproduce the issue. After merging the fix, set the the 'tryTheFix' to true and run the sample again to see the fix
var win = Ti.UI.createWindow(); win.open(); var percent_done = 0; var xhr = Ti.Network.createHTTPClient({ onload : function(e) { Ti.API.info("onLoad:"+this.responseText); }, onerror : function(e) { Ti.API.info("e.error: "+e.error); }, onreadystatechange: function(e){ Ti.API.info("onreadystatechange: "+this.readyState); }, timeout: 10000 }); function upload(){ console.log("STARTING UPLOAD"); var filename = "Wildlife.wmv"; var url = "http://www.w3schools.com/ajax/demo_post.asp"; //using this server for simplicity's sake var file = Ti.Filesystem.getFile(Ti.Filesystem.getResourcesDirectory()+filename); //just use any large-ish file, 500kb+ or so xhr.open("POST", url); // set 'tryTheFix' to true to see the fix var tryTheFix = false; var ws; if (tryTheFix) { ws = { file_name: filename, data: Ti.Utils.base64encode(file) }; } else { var toUpload = file.read(); ws = { file_name: filename, data: Ti.Utils.base64encode(toUpload) }; } xhr.send(ws); } upload();Reopening the ticket.
1. When tryTheFix = false:
We get the following errors:2. When tryTheFix = true:
We see these logs:Lokesh, The error you get with 1 may be because the server is terminating the connection because it is taking too long. Can you attach or upload the file you tried in some place? I am looking in to 2.
Hi [~sunila], The link for the attachment is : https://www.dropbox.com/s/2wt6yo16i1lto2n/wildlife.wmv
New PR submitted https://github.com/appcelerator/titanium_mobile/pull/5677
To avoid getting Broken pipe error, you could setup a local server to accept the file upload. Here is the PHP script to try
<?php set_time_limit(0); $destination_path ="temp/"; $target_path = "../htdocs/" . $destination_path; $target_path = $target_path . basename( $_FILES['data']['name']); echo "Source=" . $_FILES['data']['name'] . "<br />"; echo "Destination=" . $destination_path . "<br />"; echo "Target path=" . $target_path . "<br />"; echo "Size=" . $_FILES['data']['size'] . "<br />"; if(move_uploaded_file($_FILES['data']['tmp_name'], $target_path)) { echo "The file ". basename( $_FILES['data']['name']). " has been uploaded"; } else{ echo "There was an error uploading the file, please try again!"; } ?>3.3.X PR: https://github.com/appcelerator/titanium_mobile/pull/5733
Verified fixed on: Mac OSX 10.9.3 Appcelerator Studio, build: 3.3.0.201405271647 Titanium SDK, build: 3.3.0.v20140528144113 Titanium CLI, build: 3.3.0-beta Alloy: 1.4.0-beta Android Device: Nexus 4 (4.4) Large file successfully uploaded with base64 encoding. Closing.
update doc: https://github.com/appcelerator/titanium_mobile/pull/5942