I'm using Google Console to upload a large file (~20 GB). It appears to create a large temp file on my system disk while it's uploading, which fills up the system disk. Is it supposed to do that? Why? Is there some way I can put the temp file on another disk? I have a disk mounted with several hundred GB free, but it apparently is not used.
I'm new at using Google Cloud Storage, so if this is common knowledge, then I apologize. I did look in the Google Docs and on Stackoverflow [google-cloud-storage]. I'm on MacOS 10.12.1. Here's what happens:
On console.cloud.google.com, I open Cloud Storage and click on a bucket.
I click on "Upload Files:" and select a sparseimage of about 20 GB. The "Uploading" msg appears.
Things sit for maybe 15-20 minutes, and then MacOS says that the system is low on space. If I watch my system disk during the upload, its free space will go from about 19 GB down to a few hundred MB, then I get the system message.
After a while longer, Google Console will say that the upload is done. However, file has 0 bytes. Safari is almost unusable at this point, but I can cancel the upload (eventually), and Safari will come back to life (eventually), and I can quit it.
However, the temp file will remain until I restart MacOS. It will be something like this (the random characters vary): /private/var/folders/t9/wjkgn8c11bb0h5ncfxgwfj3h0000gn/T/WebKitGeneratedFileITn3fU
I have not installed gsutil yet; I wanted to try the simple way first.
Herman
More info:
I uploaded my 20 GB sparseimage using gsutil, and that worked just fine. It was MUCH faster than the console, by almost an order of magnitude, and it didn't create any big temp files on my system disk.
However, I had to add the -r flag and upload it as a directory. The way MacOS stores sparse image files, they look like single files on the Mac side, but they are folders, metadata, and files on the Unix side.
I looked at the 2 GB file that I uploaded previously. The type was 'application/zip', so I guess the console recognized it as a spase image and zipped it for me. I didn't tell it to. Perhaps that's what the big temp file is used for.
I can't find anything right away about automatic zipping, but more experimentation is in order.
It looks like the WebKitGeneratedFile temporary file in /private/var/folders is something that Safari does, and isn't specific to the Cloud Console, based on a thread from the Apple discussion forums.
I'd try another browser, like Google Chrome, to see if you still have the same issue.
Related
It is very strange indeed. I've uploaded several times trying different techniques and I've never had this problem before. It is also very random. I've uploaded the same photos to different named folders to test out the randomness and one photo is ok one time and the next time it is a blue blob and corrupted. Super random. I tried my Uploading Program (Downloader Pro) (never had any problems before) and it seems like it corrupted more files then just uploading the files straight from the card to the drive. I tested this and it appears to be true. More files are randomly corrupted using the Program I've always used vs just straight from the card to a folder on the drive. But this isn't the solution to just go straight from card to folder because there are still some files being randomly corrupted. What the hell right? This all of a sudden just started to happen. Also when I look at all the photos on my camera directly from the card there is never any corrupted photos. Only after I download to my computer or drive. Also I've tried with several different CF Cards and even with different Cameras. Nothing changes. Still have random corrupted files on transfer. Windows 10 Pro. Any help?
It was the Card Reader. Wow that took a lot of different testing, but as it turns out it is simply the Card Reader and nothing more. I tried everything, but that and it never occurred to me to skip the card reader and try directly from my camera. Well it works just fine straight from my camera using both my download program and the drag and drop windows style. So now I'll get a new Card Reader and I'm sure that will solve the problem and if not well I did find a work around going directly from the camera.
I'm trying to save some screen dumps to internal storage for debugging purposes, but I can't seem to get access to them. When I call FileSystemStorage.getInstance().getAppHomePath(), I get a path that looks something like this:
/data/data/com.mycompany.myapp/files/
But I can't see this folder in the Android File Transfer tool, so I can't drag the files to my Mac. I also tried attaching them to an email using the Message class, but for some reason the attachments never showed up. I notice that a lot of applications store data in folders like this:
/Android/data/com.doubletwist.androidplayer/
If I try to create a folder like this, I run into two problems. First, it's not platform independent. (This doesn't matter much because I'm just doing this for debugging.) Second, it doesn't work. I get an error telling me I need to use the directory returned by FileSystemStorage.getInstance().getAppHomePath()
Is there any way I can save files to a folder that I can actually retrieve them from? It would be more helpful if I had a platform-independent way, but any way that works is fine for now.
File system is a very "unportable" notion. By default app home is a private folder which some mobile OS's including Android 4+ keep private and inaccessible.
Android has a concept of "sdcard" which used to be a physical storage where you could write anything in any directory without a problem. This is no longer applicable for later versions of Android but you can read from the sdcard directory and detect it.
FileSystemStorage has an API to get roots and their types, if you have an sdcard type you can read from there. You can use the FileTree to see the file hierarchy as exposed to your application which can be useful for debugging.
I need to save a file in Flash without a prompt; what my program does is it gets all the frames from the stage and then it saves them as png files, along with a text file that has the name of the object, and some other properties about it. The code that I have does save it without any problems, but I need it to not prompt me, because I have lots of frames to do this with.
Is there a way to do this with Flash the program or actionscript?
No, unless you're using Adobe AIR. The reason for this is Flash Player and its programs are generally run through a browser over the Internet, and if people could use Flash Player to just start saving files on other people's computers, there would be some very serious security issues. AIR, on the other hand, is generally run on a desktop, laptop, or mobile device, and its programs are run directly off the same, having been pre-installed. So whereas a website can suddenly just start running a script with Flash Player without asking you first, AIR requires you to have already installed the script/program on your computer, meaning that it should be there intentionally. So security restrictions are lighter, enabling the use of the File class in your programs.
The only way to save a local file with the Flash Player (in the browser) is with FileReference and it will always prompt the user.
However, using AIR (desktop or mobile app) you can save to the local file system without user input, using File and FileStream. You can create an AIR app using Flash Pro, usually without any code changes other than the AIR APIs you need (flash.filesystem in this case).
Another idea, if you must use Flash Player and not AIR, is to first zip all the PNGs and only save to file after they are all packaged. This way there's only one file and prompt to save.
I have created a Storage>Blob>Container and added a file to it using an Explorer Tool.The problem is that the file download cannot be resumed,once paused
The file needs to be downloaded from the starting.Is this feature not available on Azure
It must be a combination of BLOB storage and the software doing the downloading. I tested this by using Internet Explorer to pull down a public file in BLOB storage and paused it. When attempting to resume it simply restarted as #techno stated; however, when using Azure Management Studio I was able to pause and resume with no issues.
Also, when I used Chrome I could pause and resume download from the same file as used above, but not every time. Sometimes it would restart. The browsers are looking at the partially downloaded file in the temp folders they use and looking at the incoming data from the server to determine how to ask for the remaining portion of the file. If for whatever reason the specific restart point can't be located the browser will just ask for the full file again.
Interestingly enough, when pulling down mp3 files IE was happy to let me pause, but never restarted correctly (Chrome did), but when I pulled down a zip file IE wouldn't even enable the Pause button (again, Chrome did and it worked fine).
As #siva.k points out the Azure BLOB API (both REST and SDKs) support requesting block ranges so in theory the Storage system itself supports not only resume but partial retrievals of any type.
I have some automated test (using CUnit) which require a "disk-image"-file (raw copy of a disk) to be "mounted" in windows and explored. I have previously used a tool/library called "FileDisk-17" , but this doesn't seem to work on my Windows 7 (64bit).
Update
I should point out, that changing the image-format (to say VHD) is not at option.
Any suggestions as to other (perhaps better supported) tools or libraries for mouting the file? The project is coded in ANSI C and compiled using MinGW.
Best regards!
Søren
Edit: Searching Bing for +filedisk 64 brings up a 64bit build of FileDisk, the utility you refer to:
http://www.winimage.com/misc/filedisk64.htm
And FileDisk-15 signed for 64bit here:
http://www.acc.umu.se/~bosse/
I can't vouch for it as I have never used it and am not familiar with the author.
Alternatively:
If you have a VHD, you can mount that in windows easily:
http://technet.microsoft.com/en-us/library/cc708295(WS.10).aspx
See also:
http://www.petri.co.il/mounting-vhd-files-with-vhdmount.htm
Since you have a raw DD image not a VHD, you will need to convert it first:
http://www.bebits.com/app/4554
Or qemu-img.exe can also do this:
qemu-img.exe convert -f raw rawdisk.img -O vpc rawdisk.vhd
Alternatively, you can create an empty VHD, and use DD to copy the raw image to the VHD, by opening the VHD as a raw device.
I faced this problem recently and found ImDisk to be an extremely nice solution:
Free, with source available and a very flexible open source license
Trivial setup (I have seen filedisk64 (in the accepted answer) described as having a "technical" setup)
Straightforward GUI and command-line access
Worked on Windows 7 64-bit
Seems to happily mount any kind of filesystem recognised by Windows (in my case, FAT16)
Works with files containing
Raw partitions
Entire raw disks (i.e. including the MBR and one or more partitions; which partition to mount can be selected)
VHD files (which it turns out are just raw partitions or disks with a 512-byte footer appended!)
Also can create RAM drives -- either initially empty or based on an existing disk image! (Very neat I must say!)
I did encounter minor issues trying to unmount drives. I was unable to unmount a drive from the GUI right-click context menu as the drive appeared to be "in use" by the explorer.exe process. Closing the Explorer window and using imdisk -d -m X: also didn't work; however imdisk -D -m X: (-D "forces" an unmount, whatever that means) did. This worked even if the drive was visible in an open Explorer window, without appearing to create any problems. However even after the drive appeared to have fully unmounted, an imdisk -l to list all available devices would still report that \Device\ImDisk0 exists, and if you remount the drive later, both that and \Device\ImDisk1 will appear in the output of imdisk -l (and so on with more unmount/remount cycles). This didn't create any problems with actually using the mounted drive when I tried a few unmount/remount cycles, though it theoretically might if you perform this many times between reboots.
ImDisk was invaluable for transferring the contents of a 1.5Gb disk drive with one FAT16 DOS partition from an ancient 486 machine.