I'm trying to scan the list of available networks and enumerate the security type for each SSID. I'm at a point where I can issue a SIOCSIWSCAN ioctl and parse the results. However, when I try to differentiate between a WEP network and an open network, I seem to be getting the same type of IE from the AP.
For example, I configured my Dlink DIR-655 router to be of open type vs WEP.
Since the network is a WEP network, I look for the hex byte 0xDD to tell me that this is an IE describing a WPA/WEP/open network. For this case, I only get one byte that says 0xDD for both open and WEP networks and the corresponding IE looks the same for both:
DD 18 00 50 F2 02 01 01 83 00 03 A4 00 00 27 A4 00 00 42 43 5E 00 62 32 2F 00
Does that mean that the router doesn't populate information about open networks under the byte 0xDD and I should be looking somewhere else?
PS: I've been reverse engineering the source from iwlist to tell me how to read the IEs returned. But they only seem to be describing WPA and WPA2 networks
It's not a duplicate. I've read em all.
I have a Nokia-N8-00. It's music player supports USLT (UnSynchronised Lyrics/Text). I use a tool called spotdl (https://github.com/Ritiek/Spotify-Downloader) that fetches song titles from "spotify" and downloads them from other sources (generally youtube) and merges metadata as well.
The problem is then, the music downloaded by that tool have lyrics on all my devices except N8. Fortunately, I got a music that had embedded lyrics that is supported on my phone too. I then analyzed both the files and found that in binary sequence, they have a very little difference (just for USLT section but they are different songs). The differences are :-
The one that supports :
55 53 4C 54 00 00 0A 56 00 00 03 58 58 58
The one that doesn't :
55 53 4C 54 00 00 07 38 00 00 01 58 58 58
(These sequences are for "USLT" declaration in the file)
I think it's an encoding difference. If I am right, what encoding is present and in which one? If it's not encoding, what is it?
I know these sequences can't elaborate the situation. So, here are the files I'm trying https://github.com/gaurav712/music.
I don't need supported USLT, I am just curious about it as I wanna make an implementation of it in C (I don't need language specific help though).
Here is what I got:
55 53 4C 54
Translates to:
USLT
So we got that right. Now, I believe we can merge that result with this answer:
Frame ID $xx xx xx xx (four characters)
Size $xx xx xx xx
Flags $xx xx
Encoding $xx
Text
(Taken from: ID3v2 Specification)
(or see this: https://web.archive.org/web/20161022105303/http://id3.org/id3v2-chapters-1.0)
Now, I couldn't get this from the source (because the site was down) but there is also this:
Encoding flag explanation:
• $00 ISO-8859-1 [ISO-8859-1]
• $01 UTF-16 [UTF-16]
• $02 UTF-16BE [UTF-16]
• $03 UTF-8 [UTF-8]
So, according to these findings (which I'm not too sure about), the one that is supported is UTF-8 encoded and the one not supported is UTF-16.
EDIT
I've downloaded and viewed your mp3 files for further inspection. Here are my new findings:
First of all, we were correct about the encodings:
UTF-8 is on supported:
UTF-16 is on unsupported:
Does this mean you can just turn '01' into '03' and it'll magically work? I doubt. It depends on the driver. What if the driver sees '\x00' bytes and iterprets it as end of string (as in end of USLT payload). To test this, you can try manually converting the encoding on the file (by removing extra bytes).
Secondly, running eyeD3 on linux on both files, I recovered that:
supported.mp3 -> ID3 v2.4
unsupported.mp3 -> ID3 v2.3
Perhaps that's an issue?
Also, note that the location of USLT tag in both files are different:
supported.mp3:
unsupported.mp3:
On linux, there are further tools to give you extra information, if need be:
mp3info, id3info, id3tool, exiftool, eyeD3, lltag
Are a couple examples. However, I think the main problem is in the text encoding. I was able to recover the lyrics quite fine using the above tools. But some of the tools give different answers because of ID3 version being different and so on.
This file works fine (UTF-8):
$ cat ok.txt
291054 Ţawī Rifā
This file causes an error (UTF-8):
$ cat bad.txt
291054 Ţawī Rifā‘
Here's the message:
$ freebcp 'DB.dbo.table' in bad.txt ... -c
Starting copy...
Msg 20050, Level 4
Attempt to convert data stopped by syntax error in source field
Msg 4895, Level 16, State 2
Server '...', Line 1
Unicode data is odd byte size for column 2. Should be even byte size.
Msg 20018, Level 16
General SQL Server error: Check messages from the SQL Server
The only difference is the last character, which is unicode 2018 (left single quotation mark)
Any idea what is causing this error?
The SQL Server uses UTF-16LE (though TDS starts with UCS-2LE and switches over I believe)
The column in question is nvarchar(200)
Here's the packet sent right before the error:
packet.c:741:Sending packet
0000 07 01 00 56 00 00 01 00-81 02 00 00 00 00 00 08 |...V.... ........|
0010 00 38 09 67 00 65 00 6f-00 6e 00 61 00 6d 00 65 |.8.g.e.o .n.a.m.e|
0020 00 69 00 64 00 00 00 00-00 09 00 e7 90 01 09 04 |.i.d.... ...ç....|
0030 d0 00 34 04 6e 00 61 00-6d 00 65 00 d1 ee 70 04 |Ð.4.n.a. m.e.Ñîp.|
0040 00 13 00 62 01 61 00 77-00 2b 01 20 00 52 00 69 |...b.a.w .+. .R.i|
0050 00 66 00 01 01 18 - |.f....|
Update: This issue has apparently been fixed in FreeTDS v1.00.16, released 2016-11-04.
I can reproduce your issue using FreeTDS v1.00.15. It definitely looks like a bug in freebcp that causes it to fail when the last character of a text field has a Unicode code point of the form U+20xx. (Thanks to #srutzky for correcting my conclusion as to the cause.) As you noted, this works ...
291054 Ţawī Rifā
... and this fails ...
291054 Ţawī Rifā‘
... but I found that this also works:
291054 Ţawī Rifā‘x
So, an ugly workaround would be to run a script against your input file that would append a low-order non-space Unicode character to each text field (e.g., x which is U+0078, as in the last example above), use freebcp to upload the data, and then run an UPDATE statement against the imported rows to strip off the extra character.
Personally, I would be inclined to switch from FreeTDS to Microsoft's SQL Server ODBC Driver for Linux, which includes the bcp and sqlcmd utilities when installed using the instructions described here:
https://gallery.technet.microsoft.com/scriptcenter/SQLCMD-and-BCP-for-Ubuntu-c88a28cc
I just tested it under Xubuntu 16.04, and although I had to tweak the procedure a bit to use libssl.so.1.0.0 instead of libssl.so.0.9.8 (and the same for libcrypto), once I got it installed the bcp utility from Microsoft succeeded where freebcp failed.
If the SQL Server ODBC Driver for Linux will not work on a Mac then another alternative would be to use the Microsoft JDBC Driver 6.0 for SQL Server and a little bit of Java code, like this:
connectionUrl = "jdbc:sqlserver://servername:49242"
+ ";databaseName=myDb"
+ ";integratedSecurity=false";
String myUserid = "sa", myPassword = "whatever";
String dataFileSpec = "C:/Users/Gord/Desktop/bad.txt";
try (
Connection conn = DriverManager.getConnection(connectionUrl, myUserid, myPassword);
SQLServerBulkCSVFileRecord fileRecord = new SQLServerBulkCSVFileRecord(dataFileSpec, "UTF-8", "\t", false);
SQLServerBulkCopy bulkCopy = new SQLServerBulkCopy(conn)) {
fileRecord.addColumnMetadata(1, "col1", java.sql.Types.NVARCHAR, 50, 0);
fileRecord.addColumnMetadata(2, "col2", java.sql.Types.NVARCHAR, 50, 0);
bulkCopy.setDestinationTableName("dbo.freebcptest");
bulkCopy.writeToServer(fileRecord);
} catch (Exception e) {
e.printStackTrace(System.err);
}
This issue has nothing to do with UTF-8 given that the data being transmitted, as shown in the transmission packet (bottom of the question) is UTF-16 Little Endian (just as SQL Server would be expecting). And it is perfectly good UTF-16LE, all except for the missing final byte, just like the error message implies.
The problem is most likely a minor bug in freetds that incorrectly applies logic meant to strip off trailing spaces from variable length string fields. There are no trailing spaces, you say? Well, if it hadn't gotten chopped off then it would be a little clearer (but, if it hadn't gotten chopped off there wouldn't be this error). So, let's look at what the packet to see if we can reconstruct it.
The error in the data is probably being overlooked because the packet contains an even number of bytes. But not all fields are double-byte, so it doesn't need to be an even number. If we know what the good data is (prior to the error), then we can find a starting point in the data and move forwards. It is best to start with Ţ as it will hopefully be above the 255 / FF value and hence take 2 bytes. Anything below will have a 00 and many of the characters have that on both sides. While we should be able to assume Little Endian encoding, it is best to know for certain. To that end, we need at least one character that has two non-00 bytes, and bytes that are different (one of the character is 01 for both bytes and that does not help determine ordering). The first character of this string field, Ţ, confirms this as it is Code Point 0162 yet shows up as 62 01 in the packet.
Below are the characters, in the same order as the packet, their UTF-16 LE values, and a link to their full details. The first character's byte sequence of 62 01 gives us our starting point, and so we can ignore the initial 00 13 00 of line 0040 (they have been removed in the copy below for readability). Please note that the "translation" shown to the right does not interpret Unicode, so the 2-byte sequence of 62 01 is displayed as 62 by itself (i.e. lower-case Latin "b") and 01 by itself (i.e. non-printable character; displayed as ".").
0040 xx xx xx 62 01 61 00 77-00 2b 01 20 00 52 00 69 |...b.a.w .+. .R.i|
0050 00 66 00 01 01 18 ?? - |.f....|
Ţ -- 62 01 -- http://unicode-table.com/en/0162/
a -- 61 00 -- http://unicode-table.com/en/0061/
w -- 77 00 -- http://unicode-table.com/en/0077/
ī -- 2B 01 -- http://unicode-table.com/en/012B/
-- 20 00 -- http://unicode-table.com/en/0020/
R -- 52 00 -- http://unicode-table.com/en/0052/
i -- 69 00 -- http://unicode-table.com/en/0069/
f -- 66 00 -- http://unicode-table.com/en/0066/
ā -- 01 01 -- http://unicode-table.com/en/0101/
‘ -- 18 20 -- http://unicode-table.com/en/2018/
As you can see, the last character is really 18 20 (i.e. a byte-swapped 20 18 due to the Little Endian encoding), not 01 18 as it might appear if reading the packet starting at the end. Somehow, the final byte -- hex 20 -- is missing, hence the Unicode data is odd byte size error.
Now, 20 by itself, or followed by 00, is a space. This would explain why #GordThompson was able to get it working by adding an additional character to the end (the final character was no longer trimmable). This could be further proven by ending with another character that is a U+20xx Code Point. For example, if I am correct about this, then ending with ⁄ -- Fraction Slash U+2044 -- would have the same error, while ending with ⅄ -- Turned Sans-Serif Capital Y U+2144 -- even with the ‘ just before it, should work just fine (#GordThompson was kind enough to prove that ending with ⅄ did work, and that ending with ⁄ resulted the same error).
If the input file is null (i.e. 00) terminated, then it could simply be the 20 00 ending sequence that does it, in which case ending with a newline might fix it. This can also be proven by testing a file with two lines: line 1 is the existing row from bad.txt, and line 2 is a line that should work. For example:
291054 Ţawī Rifā‘
999999 test row, yo!
If the two-line file shown directly above works, that proves that it is the combination of a U+20xx Code Point and that Code Point being the last character (of the transmission more than of the file) that exposes the bug. BUT, if this two-line file also gets the error, then it proves that having a U+20xx Code Point as the last character of a string field is the issue (and it would be reasonable to assume that this error would happen even if the string field were not the final field of the row, since the null terminator for the transmission has already been ruled out in this case).
It seems like either this is a bug with freetds / freebcp, or perhaps there is a configuration option to not have it attempt trimming trailing spaces, or maybe a way to get it to see this field as being NCHAR instead of NVARCHAR.
UPDATE
Both #GordThompson and the O.P. (#NeilMcGuigan) have tested and confirmed that this issue exists regardless of where the string field is in the file: in the middle of a row, at the end of the row, on the last row, and not on the last row. Hence it is a general issue.
And in fact, I found the source code and it makes sense that the issue would happen since there is no consideration for multi-byte character sets. I will file an Issue on the GitHub repository. The source for the rtrim function is here:
https://github.com/FreeTDS/freetds/blob/master/src/dblib/bcp.c#L2267
Regarding this statement:
The SQL Server uses UTF-16LE (though TDS starts with UCS-2LE and switches over I believe)
From an encoding stand-point, there is really no difference between UCS-2 and UTF-16. The byte sequences are identical. The only difference is in the interpretation of Surrogate Pairs (i.e. Code Points above U+FFFF / 65535). UCS-2 has the Code Points used to construct Surrogate Pairs reserved, but there was no implementation at that time of any Surrogate Pairs. UTF-16 simply added the implementation of the Surrogate Pairs in order to create Supplementary Characters. Hence, SQL Server stores and retrieves UTF-16 LE data without a problem. The only issue is that the built-in functions don't know how to interpret Surrogate Pairs unless the Collation ends with _SC (for Supplementary Characters), and those Collations were introduced in SQL Server 2012.
This might be an encoding issue of the source file.
As you are using non-standard characters, the source file should be unicode by itself probably. Other encodings use a differing count of bytes (one up to three) to encode one single character. E.g. your Unicode 2018 is 0xE2 0x80 0x98 in UTF-8.
Your packet ends with .R.i.f....| while there should be your ā‘. And the error shows Server '...', Line 1.
Try to find out the encoding of your source file (look at big and little endian too) and try to convert your file to a sure unicode format.
This might solve it:
inf your /etc/freetds/freetds.conf
add:
client charset = UTF-8
also found this about the flag use utf-16
use utf-16 Instead of using UCS-2 for database wide
character encoding use UTF-16. Newer Windows versions use this
encoding instead of UCS-2. This could result in some issues if clients
assume that a character is always 2 bytes.
If i buy 7 xbee-zigbee. I setup xbee modules like this.
1 coordinator (C)
3 router (R1-R2 -R3)
3 end device (ED1 - ED2 - ED3)
Can a router scan all end device with "AS" (active scan) command?
Result must be like this;
R1 get rssi value from ED1 : -90 dbm
R1 get rssi value from ED2 : -60 dbm
R1 get rssi value from ED3 : -80 dbm
R2 get rssi value from ED1 : -40 dbm
R2 get rssi value from ED2 : -55 dbm
R2 get rssi value from ED3 : -89 dbm
R3 get rssi value from ED1 : -23 dbm
R3 get rssi value from ED2 : -10 dbm
R3 get rssi value from ED3 : -70 dbm
Is this possible? Or is there another way?
Devices will only respond to a Beacon Request (Active Scan) if they are in the "RX on when idle" configuration. This mostly means Routers (and the Coordinator).
There are a few different types of End Device. If they indicates in their Device Announce message that they are "RX on when idle" then they should respond to Beacon Requests. Very few devices actually work this way (a few light bulbs). Most End Devices are battery powered and will not respond to Beacon Requests because they aren't on to hear them.
The ATAS command was designed for network discovery, and you use it when you aren't connected to an active network. The XBee module will scan multiple channels for networks and dump information on each beacon it receives on those channels. Since you can't join a network through an end device, you're not going to see beacons coming from those devices.
You might be more interested in exploring the ATND (Node Discovery) command, for getting a list of devices on the current network.
Mvut from digi said this:"you can use the S2C ZB module and use the ZDO commands to report the Link status." Any body try this?
you can try API mode , it can sent Remote AT command(0x17) and sent back response(0x97) .
something like:
7E 00 0F 17 01 00 00 00 00 00 00 FF FF FF FF 02 64 62 23
0x7E - package start
0x00、0x0F - package length
0x17 - frame type
0x01 - ! this byte is important about if you want to get the response or not
...
I hope it can help for you~
I have two XBee chips - one of them is connected to a relay switch, the other one - to my computer via USB cable.
I can configure the locally connected XBee to send its D0 value to the remote XBee, so that when I toggle the D0 line of the local XBee the remote relay switch toggles as well.
What I want to do is be able to send a command to the local XBee over its serial connection and have the local XBee send a command to the remote XBee that would toggle the relay switch without having to physically interact with the D0 line on my local XBee.
The XBees are S1, so they don't (seem to?) support ATIO command, at least my tests didn't show that working. I also tried using ATAP 1 with API command 83 as shown here but that didn't work.
The hardware setup works - attaching a button to D0 transmits its status to the remote XBee, so how do I get the same to happen with software alone?
You need to send a "Remote AT Command" frame, for parameter ATD0, as described in this page on Digi's website.
Although that page is for the Series 2 radio modules, if you look at the documentation for the Series 1, you can find the frame format for a remote AT command.
And, if you're going to use C to send the command, this Open Source, portable, ANSI C XBee Host Library includes a function process_command_remote() in samples/common/_atinter.c to send a remote AT command.
Finally figured it out, thanks for steering me in the right direction #tomlogic
The problem was that Digi's website doesn't tell you to set IA to 0xFFFF (allow all source addresses to change pin state), which by default is 0xFFFF FFFF FFFF FFFF (disable remote pin changes)
Found a better tip on this site.
This is what it looks like in XCTU for Mac:
Here are all the settings that differ from defaults one I got it to work.
Transmitter:
MY=7
AP=1 (API enabled)
D0=3 (Pin 0 Input)
IC=FF (Change detect all)
Receiver:
MY=2
D0=5 (Pin 0 High Output)
IU=0 (UART IO Disable)
IA=0xFFFF (Allow all to change pins)
The commands I used:
7E 00 10 17 01 00 13 A2 00 AA BB CC DD FF FE 02 49 4F 00 8D - Send remote ATIO 0
7E 00 10 17 01 00 13 A2 00 AA BB CC DD FF FE 02 49 4F 01 8C - Send remote ATIO 1