I think I understand this correctly, but I just want to double check to be sure. Suppose I write a binary file using VBA where the first X number of bytes represents some field, the next x number another field, and so on. Now suppose I read that binary file back into VBA later using a byte array. Is it reasonable to assume that the first x elements in the byte array directly correlate to the first x bytes in the file?
I should have made this clear from the get-go, the format and header of the file isn't all that important, I'm just trying to get more into the nitty-gritty of reading and writing binary files and using byte arrays with vba. I'm getting there, and I appreciate everyone's input.
When writing binary files, write headers to check if the given bytes are really matching your format. Take a look into a wave file with a hex editor (i use HxD) and you will see something like this:
RIFFŽ...WAVEfmt ........D¬...±......dataà€....
RIFF is the header of the container (resource interchange
format)
followed by some bytes for meta information
WAVE is the
header of the actual wave
then some data follows you might want to
interprete
Here are two example binary read/write methods (converted to VB.NET from C#)
Public Shared Sub Write(This As YOUR_TYPE, stream As BinaryWriter)
stream.Write(FILE_IDENTIFIER)
stream.Write(FILE_VERSION)
While True
stream.Write(/*...*/)
End While
End Sub
Public Shared Function read(stream As BinaryReader) As YOUR_TYPE
If Not Enumerable.SequenceEqual(FILE_IDENTIFIER, stream.ReadBytes(FILE_IDENTIFIER.Length)) Then
Throw New FormatException("header mismatch")
End If
If stream.ReadByte() <> FILE_VERSION Then
Throw New NotSupportedException("version mismatch")
End If
Dim result As New YOUR_TYPE()
While True
stream.Read(/*...*/)
End While
Return result
End Function
Related
I am trying to find an example, or documentation, on creating a picture (floating shape object) inside the Excel sheet. The source is supposed to be a numeric bitmap data, stored in a VBA array acquired using external I/O libraries. Using Excel cells as an intermediary storage is possible, but not desired, since the RGB bitmap data is expected to be huge.
The task itself seems to be extremely simple in matlab-like environments, or python. But I just have no Idea how to make it in Excel and VBA without importing an independent image file from the file system.
In terms of storing the file, how huge is 'huge'? If you convert the image into Base64, it'll be a fairly trivial task to split it up amongst the cells and then reconstitute it when converting it into an image.
Alternatively, you can store the Base64 string in a standard module - I'm currently doing much the same thing, but my image only clocks in at 100kb (better to save it as a PNG rather than BMP).
In terms of converting the Base64 string to an image, the Windows Image Acquisition COM object will convert a byte array into a stdPicture image type (and further to my point above, it will also accept PNG files...]. The following function accepts a Base64 string, converts it into a byte array, and returns an stdPicture object:
Function Base64toStdPicture(ByVal Base64Code As String) As StdPicture
Dim ImgVector As Object
Dim Node As Object
Set ImgVector = CreateObject("WIA.Vector")
Set Node = CreateObject("Msxml2.DOMDocument.3.0").createElement("base64")
Node.DataType = "bin.base64"
Node.Text = Base64Code
ImgVector.BinaryData = Node.nodeTypedValue
Set Base64toStdPicture = ImgVector.BinaryData.Picture
Set Node = Nothing
Set ImgVector = Nothing
End Function
From that point, you can out it in an image control, or copy it to / from the clipboard, etc.
I am logging data using IronPython (sensor OEM requires this), and writing that collected data to a .txt file using the first snippet below. My goal is to make real-time calculations on the collected data (like a running window variance), and plot it in real-time. I am reading the .txt file into a script I've written in Spyder.
I am losing about 3/4 of the data points somewhere between the data collection in my first script, and the data read in my Spyder script for the math calculation and visualization.
I am looking for some feedback for the way I am writing my data, and reading it; and if I am risking losing data somewhere along this way.
file = open("C:\\Users\Documents\\test4.txt", "a")
writer = csv.writer(file)
while True:
writer.writerow(myFunction())
file.flush()
In Spyder, I am reading in this file like the following:
def animate(i):
data = pd.read_csv("C:\\Users\Documents\\test4.txt")
data.columns = ['X', 'Y', 'Z']
when using the ETCD dashboard, we find a binary string like
var _deps_js = "\x1f\x8b\x08\x00\x00\x09\x6e\x88\x00\xff\xcc\xbd\x7b\x77\xdb\x46\xb2\x2f\xfa\xff\xf9\x14\x22\xc6\x47\x01\xcc\x16\x25\xd9\x99\xb9\x7b\xc0\x40\xbc\x89\x1d\x27\xce\x24\x76\x26\x76\x26\x0f\x8a\xc9\xc2\x8b\x24\x24\x92\xa0\x49\xca\x92\x23\x72\x3e\xfb\xad\x5f\x55\x77\xa3\x01\x82\x72\xf6\xec\x7b\xd6\x3a\x99\xb1\x88\x47\xa3\x9f\xd5\xd5\xf5\xae\xce\xf8\x66\x91\x6e\x8a\x72\xe1\xc7\x2a\x09\xee\xbd\x32\xb9\xca\xd3\x8d\x17\x45\x9b\x0f\xcb\xbc\x1c\x1f\xcd\xcb\xec\x66\x96\x1f\x1f\x1f\x78\xd1\xcb\xef\x96\xe5\x6a\xb3\x1e\xd4\x6f\xa3\xb8\x97\x95\xe9\xcd\x3c\x5f\x6c\x06\x09\xd5\xdc\x39\x0b\xc2\xaa\xa1\xe0\xbe\x18\xfb\x9d\xaa\x48\xb0\x99\xae\xca\xdb\xa3\x45\x7e\x7b\xf4\xe5\x6a\x55\xae\x7c\xef\xea\x9f\x37\xf9\xea\xc3\xd1\x2a\x7f\x77\x53\xac\xf2\xf5\x51\x7c\x74\x5b\x2c\x32\x2a\x73\x5b\x6c\xa6\x74\x67\xbe\xf4\x82\xfe\x2a\xdf\xdc\xac\x16\x47\xd4\x4a\xb0\x0b\xf9\xaf\xef\xdd\x2c\xb2\x7c\x5c\x2c\xf2\xcc\xeb\x98\xee\xca\xf7\x03\xf9\x09\x37\xd3\x62\xad\xea\x23\x37\x77\x47\x29\x3a\xf8\x3e\x5e\x1d\x25\x34\x8a\x59\xbe\x98\x6c\xa6\x2a\x8d\xe2\xa4\x87\x9a\xe8\x9d\x6e\xd1\x33\x1f\xd0\x94\x44\xe9\x76\x4b\x05\x8a\xf5\x4f\x5c\x3d\x15\x1a\x74\xce\xc3\x73\x7a\x11\xf7\x16\x65\x96\xbf\xa5\x2f\x8f\x8f\x93\x41\xe7\x2c\xf4\xe2\xd5\x2a\xfe\xa0\xbf\x39\xa3\x9f\x64\xbb\xf5\x16\x37\xf3\x24\x5f\x55\x73\x9b\x50\xe1\x8b\x33\xfa\x73\x72\x7e\x54\x2c\x8e\xe2\x9d\xed\x5c\x86\xce\xaa\x94\x67\x90\x5b\x7c\x61\x06\x91\x04\x81\x9e\x0a\x7a\x3e\x59\xe5\x4b\x2a\xe
it's a long list.
Is there a tool to convert this into a readable text so that I can know the exact the content.
The two leading bytes "\x1f\x8b" indicate the data is gzip compressed.
You can use whatever gzip/gunzip library/utility you have to decode it.
Following is an example using Perl.
use IO::Uncompress::Gunzip;
my $in = "\x1f\x8b..."; # <- your string goes here
&IO::Uncompress::Gunzip::gunzip( \$in => \my $out );
print $out;
and it returns something like
!function(a,b){"object"==typeof module&&"object"==typeof module.exports?module.exports=a.document?b(a,!0):function(a){if(!a.document)throw new Error("jQuery requires a window with a document");return b(a)}:b(a)}("undefined"!=typeof window?window:this,function(a,b){function c(a){var b=a.length,c=ab.type(a);return"function"===c||ab.isWindow(a)?!1:1===a.nodeType&&b?!0:"array"===c||0===b||"number"==typeof b&&b>0&&b-1 in a}function d(a,b,c){if(ab.isFunction(b))return ab.grep
Since the string in your question is truncated, above output is truncated.
Currently I am sending the UART the strings I want to log and reading it on the host with any terminal.
In order to reduce the logging time and hopefully the image size as well (my flash is tiny), I figured out that the strings are unused in the embedded system, so why storing them on the flash?
I want to implement a server, whom I can send a hashed-key of any string (for example - it's ROM address) and the string will be output to file or screen.
My questions are:
How to create the key2string converter out of the image file (the OS is CMX, but can be answered generally)
Is there a recomended way to generate image, that will know the strings addresses but will exclude them from ROM?
Is there a known generic (open-source or other) that implemented a similar logger?
Thanks
Rather than holding hard-coded strings, then trying to hash the answers and sent it via a UART, then somehow remove the strings from the resulting image, I suggest the following.
Just send an index for an error code. The PC side can look up that index and determine what the string is for that condition. If you want the device code to be more clear, the index can be an enumeration.
For example:
enum errorStrings
{
ES_valueOutOfLimits = 1,
ES_wowItsGettingWarm = 2,
ES_randomError = 3,
ES_passwordFailure = 4
};
So, if you were sending data to the UART via printf, you could do the following:
printf("%d\n",(int)ES_wowItsGettingWarm);
Then your PC software just needs to decode the "2" that comes across the UART back into a useful string of "Wow it's getting warm."
This keeps the firmware small, but you need to manually keep the file containing the enum and the file with the strings in sync.
My solution is sending file name and line (which should be 14-20 Byte) and having a source parser on the server side, which will generate map of the actual texts. This way the actual code will contain no "format" strings, but single "filename" string for each file. Furthermore, file names can be easily replaced with enum (unlike replacing every string in the code) to reduce the COMM throughput.
I hope the sample psaudo-code will help clarifying the idea:
/* target code */
#define PRINT(format,...) send(__FILE__,__LINE__,__VA_ARGS__)
...
/* host code (c++) */
void PrintComm(istream& in)
{
string fileName;
int line,nParams;
int* params;
in>>fileName>>line>>nParams;
if (nParams>0)
{
params = new int[nParams];
for (int i=0; i<nParams; ++i)
in>>params[i];
}
const char* format = FindFormat(fileName,line);
...
delete[] params;
}
I'm currently trying to attach image files to a model directly from a zip file (i.e. without first saving them on a disk). It seems like there should be a clearer way of converting a ZipEntry to a Tempfile or File that can be stored in memory to be passed to another method or object that knows what to do with it.
Here's my code:
def extract (file = nil)
Zip::ZipFile.open(file) { |zip_file|
zip_file.each { |image|
photo = self.photos.build
# photo.image = image # this doesn't work
# photo.image = File.open image # also doesn't work
# photo.image = File.new image.filename
photo.save
}
}
end
But the problem is that photo.image is an attachment (via paperclip) to the model, and assigning something as an attachment requires that something to be a File object. However, I cannot for the life of me figure out how to convert a ZipEntry to a File. The only way I've seen of opening or creating a File is to use a string to its path - meaning I have to extract the file to a location. Really, that just seems silly. Why can't I just extract the ZipEntry file to the output stream and convert it to a File there?
So the ultimate question: Can I extract a ZipEntry from a Zip file and turn it directly into a File object (or attach it directly as a Paperclip object)? Or am I stuck actually storing it on the hard drive before I can attach it, even though that version will be deleted in the end?
UPDATE
Thanks to blueberry fields, I think I'm a little closer to my solution. Here's the line of code that I added, and it gives me the Tempfile/File that I need:
photo.image = zip_file.get_output_stream image
However, my Photo object won't accept the file that's getting passed, since it's not an image/jpeg. In fact, checking the content_type of the file shows application/x-empty. I think this may be because getting the output stream seems to append a timestamp to the end of the file, so that it ends up looking like imagename.jpg20110203-20203-hukq0n. Edit: Also, the tempfile that it creates doesn't contain any data and is of size 0. So it's looking like this might not be the answer.
So, next question: does anyone know how to get this to give me an image/jpeg file?
UPDATE:
I've been playing around with this some more. It seems output stream is not the way to go, but rather an input stream (which is which has always kind of confused me). Using get_input_stream on the ZipEntry, I get the binary data in the file. I think now I just need to figure out how to get this into a Paperclip attachment (as a File object). I've tried pushing the ZipInputStream directly to the attachment, but of course, that doesn't work. I really find it hard to believe that no one has tried to cast an extracted ZipEntry as a File. Is there some reason that this would be considered bad programming practice? It seems to me like skipping the disk write for a temp file would be perfectly acceptable and supported in something like Zip archive management.
Anyway, the question still stands:
Is there a way of converting an Input Stream to a File object (or Tempfile)? Preferably without having to write to a disk.
Try this
Zip::ZipFile.open(params[:avatar].path) do |zipfile|
zipfile.each do |entry|
filename = entry.name
basename = File.basename(filename)
tempfile = Tempfile.new(basename)
tempfile.binmode
tempfile.write entry.get_input_stream.read
user = User.new
user.avatar = {
:tempfile => tempfile,
:filename => filename
}
user.save
end
end
Check out the get_input_stream and get_output_stream messages on ZipFile.