This is my second post on this subject.
4k video textures, such as 4096 x 2048, do not render on iOS or Android with three.js. The 4k video file's bitrate, frame rate and profile (baseline, main) make no difference. No combination of settings work. 4k video files simply will not render on any mobile device using three.js. Video files below 4k, such as 3000 x 1500 etc, will visually render but performance is significantly affected as the file is not a power of 2.
A significant number of tests have been made to overcome this limitation. Nothing works, however. 4k video textures don't render on either mobile browsers in a Cordova + Crosswalk developed application.
All tests have been made on relatively new devices, such as the iPad Air 2 (iOS 10.2.1) and an Android based Huawei P8 Max. Upon visiting www.webglreport.com on the iPad Air 2, it states the Max Texture Size and Max Cube Map Texture Size are 4096. So, technically three.js should handle the 4k (4096 x 2048) size.
Based on our tests, I believe this is an issue related to three.js. Any time or thoughts put on this would be tremendously appreciated as we've been dealing with this issue for quite a long time now.
Related
I'm using DatoCMS and NextJS to build a website. DatoCMS uses Mux behind the scenes to process the video.
The video that comes through is fairly well optimised for whatever browser is being used, and potentially for ABR with HLS; however, it still can take a fair bit of time on the initial load.
The JSON from Dato includes some potentially useful other things:
"video": {
"mp4Url": "https://stream.mux.com/6V48g3boltSf5uQRB8HnelvtPglzZzYu/medium.mp4",
"streamingUrl": "https://stream.mux.com/6V48g3boltSf5uQRB8HnelvtPglzZzYu.m3u8",
"thumbnailUrl": "https://image.mux.com/6V48g3boltSf5uQRB8HnelvtPglzZzYu/thumbnail.jpg"
},
"id": "44785585",
"blurUpThumb": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAoHBwgHBhUICAgLCgoXDhgVDhkNDh0VGhUeFxUZHSIfGxUmKzcjHh0oHRwWJDUlKDkvMjIyGSI4PTcwPCsxMi8BCgsLDg0OHBAQHDsoIig7NTs7Ozs7Ozs7Ozs7Ly8vOzs1Ozs7Ozs7Ozs1NTU7Ozs1Ozs7OzUvLzsvLy8vLy8vL//AABEIAA8AGAMBIgACEQEDEQH/xAAYAAACAwAAAAAAAAAAAAAAAAAEBQACBv/EAB4QAAICAgIDAAAAAAAAAAAAAAECABEEBQMGEiIy/8QAFwEAAwEAAAAAAAAAAAAAAAAAAgMFAP/EABsRAAIDAQEBAAAAAAAAAAAAAAECAAMRIUEE/9oADAMBAAIRAxEAPwDIYui48Y+saphApUQL2ZHNeJELTfALdGE943pl2m+gDFPJfP0qc/1JAMntKA0FYyTC9vIt2+JzrZP/2Q=="
}
With either next/image, or the more proprietary react-datocms/image, that blurUpThumb could be used as a placeholder while the full image is being loaded in the background, to improve UX, and (I believe) page-load speed / time to interactive.
Is there a way to achieve that same effect with the video instead of a file?
The usual way an ABR, HLS or DASH etc, video can start faster is by starting with one of the lower resolutions and stepping up to a higher resolution after the first couple of segments once the video is playing and there is more time to buffer.
However in your case, the example video is very short, 13 seconds so the effect is pretty minimal. Playing it on Safari on a MAC I saw the step happen at 4 seconds which is almost a 3rd through already in this case.
Short of re-encoding with lower resolution or some special codecs I think you may find this hard to beat - Mux is a pretty mature video streaming service.
The direct links to the videos above loaded and played quite quickly for me, even over a relatively low speed internet. It might be worth looking at what else your page is loading at the same time as this may be competing for bandwidth and slowing things down.
How can i implement a Video Player with transparent background in React-native? The video is saved with the extension .mov (only extension found to support transparent background).
Used react-native-player but it only renders a back screen.
Tried with https://www.w3schools.com/html/mov_bbb.mp4 both loaded locally and via uri and it worked.
Platform target: iOs
I needed same thing and tried many different methods and my conclusion is transparent movie file is not an accepted practice/standard. Only apple quicktime supports it.
Here are few workarounds I tried and made it partially work with a lot of performance challenges.
Best working method is to export all mov frames (with whatever frame rate is acceptible for you - for me I was trying to render something that was going to be exported as gif, so even 15 fps worked for me). Export movie frames as transparent png. Ffmpeg can easily do that.
Then either with simple loop, load and update frames from the js code. This is still very challenging because loading each frame in an image view from jsbundle simply does not catch up with your frame rate needs. As far as my experience, I only saw low single digit frame rates with loading resources from jsbundle. On the other hand, there is a solution for this. You can put your png files in xcassets in ios/xcode, and drawables in android. Then use { uri: 'filename' } to load resources from native apps. This resulted very good quality and speed for me on iOS even for 20-30 second video lengths at 1080p quality. Unfortunately this suffered from memory issues and didn't work on Android - Android only handled 100something frame for me. If you have more than 150-200 frames for your movie in total, regardless of fps, it will hit to the memory limits. I spent more than a week to research alternative ways to overcome memory issue, load many large bitmaps in memory for sequence display them in the app. There are theoretical methods to utilize memory on android without heap size limitations but I found it too low level and beyond my android/java knowledge.
Another path I experimented without success was webM format. It's webp format from google that is supported with some push but I couldn't find enough resources online about webm playback. I succeeded to play a webm file but it just works like gif players which almost all of them don't have clear ways to support transparent animated gifs. So I wasn't able to get to a solution with webp/m as well.
Right now I am trying to fake the transparent portions of my video with other RN animated elements over the video.
By the way, I tried most of these solutions natively on iOS and Android separately and both platforms had its own problems with them.
I have Windows Phone 8.1 Silverlight app which use GetThumbnailAsync method to show preview of images in Camera Roll. Almost all the time it works as expected and provide thumbnail within around 50-200 milliseconds. But for some of the images in my test device (Lumia 820), it takes around 10 seconds to return the thumbnail. However, when I copy same images to another test device (Lumia 1020), it works as expected. In Lumia 1020, still I didn't see this strange behaviour for any other image. Moreover, I didn't find any relation between this problem and size of the image. I really appreciate it if anyone can share any thought or experience on this issue.
Edit: Here is how I make the GetThumbnailAsync call in my app
var thumbnail = await selectedFile.GetThumbnailAsync(Windows.Storage.FileProperties.ThumbnailMode.SingleItem, 400)
Windows Phone 7.1/7.5/Mango Silverlight App.
Coming from here: Need clarification on using audio on button click, background audio, etc in Windows Phone
Our designer will be converting all the mp3s to .wav files. Done few and they are coming to about 200kb each.
The current estimate is we might have like 100+ of those for our app.
I know the Certification Requirment is:
The maximum size of the XAP package file is 225 MB.
Designer said he will try to compress them down to about 100kb making sure sound quality is ok as well.
Though I am sure we won't exceed 225MB but I think lesser is better as it will affect the download time on the device as well. Don't want the user to quit download halfway.
I read somewhere there is some time restiction as well for certification.
Is this acceptable, or am I missing any other strategies for keeping my audio files small other than compression? Are there any other considerations I need to take into account when certifying a large app?
Keep in mind that the number of video files and resources together shouldn't exceed 2000 files (Plus the size requirement of course). I had a lot of issues in my experience with submitting xap packages that contains a lot of files. The last app was a video dictionary that contains more than 2000 video files all with tiny size but that didn't work well, though the size was just 90 Mega bytes, the responses from the support are slow and we had to wait each time to finally find that we had to respect this rule which is not documented
IMO, download times don't largely affect conversion rates, because they download in the background. I'll frequently download a few apps, then check back on them the next day or so.
So I know that App Engine prevents working with images greater than 1 MB in size, but I'm getting a RequestTooLargeError when I call images.resize on an jpg that is 400K on disk. The dimensions of the jpg are 1600 x 1200, so is it that app engine can't handle resizing images over 1 megapixel, even if the image file itself is a compressed format that is smaller than 1 MB?
This a is best guess... not an real answer.
Based on what I have read here and in some other threads, it seems like the image api has decompressed your image into a form that is larger than 1 MB and then proceeded to complain about the image that it created.
About the only way to prevent that is to cut your original image into chunks that will not be bigger than 640x520... But that will require some pretty heavy listing on the client side.
Added: This app engine issue regarding image size limits may have some helpful pointers
Added: You can probably leverage the finding that you had in your initial revision of this question... you said that crop worked but resize did not... This will allow you to keep most of the processing on the server-side.
Added: Another thread about the effects of small JPG that transforms into a larger image
The image API would not raise a RequestTooLargeError (see exceptions related to image API here). It would indicate that your total request size is too big.
What other data are you sending over with the request? Although it probably would be hard to push the total request over 10MB (which is the maximum request size) if it's a fairly simple request (i.e. just uploading a single image).