Unable to boot: Buffer I/O error on /dev/sda4, logical block 0 - filesystems
I am unable to boot into Ubuntu 22.04 and am shown the message "Buffer I/O error on /dev/sda4". sda4 is the HDD partition where my home directory is mounted.
After referring to a few similar questions, I have run some of the suggested commands and attached their output:
$ fsck -a /dev/sda4
fsck from util-linux 2.38.1
/dev/sda4:
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
Running e2fsck gave me similar errors.
$ mount /dev/sda4 /mnt/temp
mount: /mnt/temp: can't read superblock on /dev/sda4.
dmesg(1) may have more information after failed mount system call.
$ smartctl -a /dev/sda4
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.0.7-301.fc37.x86_64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Seagate Mobile HDD
Device Model: ST1000LM035-1RK172
Serial Number: WQ9AWWCF
LU WWN Device Id: 5 000c50 0d4515054
Firmware Version: LFM3
User Capacity: 1,000,204,886,016 bytes [1.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5400 rpm
Form Factor: 2.5 inches
TRIM Command: Available
Device is: In smartctl database 7.3/5319
ATA Version is: ACS-3 T13/2161-D revision 3b
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sun Feb 12 04:03:09 2023 IST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
See vendor-specific Attribute list for marginal Attributes.
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x71) SMART execute Offline immediate.
No Auto Offline data collection support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 169) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x3035) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 036 033 034 Pre-fail Always In_the_past 165182416
3 Spin_Up_Time 0x0003 100 100 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 094 094 020 Old_age Always - 6158
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 8
7 Seek_Error_Rate 0x000f 069 060 045 Pre-fail Always - 17210113963
9 Power_On_Hours 0x0032 095 095 000 Old_age Always - 5092 (225 1 0)
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 097 097 020 Old_age Always - 3985
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 18151
188 Command_Timeout 0x0032 100 096 000 Old_age Always - 4295491592
189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 067 047 040 Old_age Always - 33 (Min/Max 30/39)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 98
192 Power-Off_Retract_Count 0x0032 099 099 000 Old_age Always - 3234
193 Load_Cycle_Count 0x0032 083 083 000 Old_age Always - 34491
194 Temperature_Celsius 0x0022 033 053 000 Old_age Always - 33 (0 24 0 0 0)
196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 8
197 Current_Pending_Sector 0x0012 099 098 000 Old_age Always - 96
198 Offline_Uncorrectable 0x0010 099 098 000 Old_age Offline - 96
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x000f 096 096 030 Pre-fail Always - 3831 (52 158 0)
254 Free_Fall_Sensor 0x0032 100 100 000 Old_age Always - 0
SMART Error Log Version: 1
ATA Error Count: 18120 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 18120 occurred at disk power-on lifetime: 5091 hours (212 days + 3 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 02:36:35.548 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 00 02:36:35.538 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 02:36:35.512 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 02:36:35.511 IDENTIFY DEVICE
ef 03 45 00 00 00 a0 00 02:36:35.498 SET FEATURES [Set transfer mode]
Error 18119 occurred at disk power-on lifetime: 5091 hours (212 days + 3 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 02:36:33.822 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 02:36:33.822 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 02:36:33.821 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 00 02:36:33.802 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 02:36:33.775 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
Error 18118 occurred at disk power-on lifetime: 5091 hours (212 days + 3 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 02:36:32.075 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 00 02:36:32.065 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 02:36:32.039 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 02:36:32.037 IDENTIFY DEVICE
ef 03 45 00 00 00 a0 00 02:36:32.025 SET FEATURES [Set transfer mode]
Error 18117 occurred at disk power-on lifetime: 5091 hours (212 days + 3 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 02:36:30.350 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 00 02:36:30.340 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 02:36:30.313 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 02:36:30.312 IDENTIFY DEVICE
ef 03 45 00 00 00 a0 00 02:36:30.300 SET FEATURES [Set transfer mode]
Error 18116 occurred at disk power-on lifetime: 5091 hours (212 days + 3 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 20 ff ff ff 4f 00 02:36:28.246 READ FPDMA QUEUED
ea 00 00 00 00 00 a0 00 02:36:28.243 FLUSH CACHE EXT
ef 10 02 00 00 00 a0 00 02:35:23.920 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 02:35:23.893 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 02:35:23.892 IDENTIFY DEVICE
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 5091 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
$ fdisk -l /dev/sda4
fdisk: cannot open /dev/sda4: Input/output error
$ lsblk
loop0 7:0 0 1.8G 1 loop
loop1 7:1 0 7.6G 1 loop
├─live-rw 253:0 0 7.6G 0 dm /
└─live-base 253:1 0 7.6G 1 dm
loop2 7:2 0 32G 0 loop
└─live-rw 253:0 0 7.6G 0 dm /
sda 8:0 0 931.5G 0 disk
├─sda1 8:1 0 128M 0 part
├─sda2 8:2 0 540.8G 0 part
├─sda3 8:3 0 7.6G 0 part
├─sda4 8:4 0 279.4G 0 part
└─sda5 8:5 0 103.6G 0 part
sdb 8:16 1 28.7G 0 disk
└─sdb1 8:17 1 28.7G 0 part /run/initramfs/live
zram0 252:0 0 7.4G 0 disk [SWAP]
nvme0n1 259:0 0 119.2G 0 disk
├─nvme0n1p1 259:1 0 260M 0 part
├─nvme0n1p2 259:2 0 16M 0 part
├─nvme0n1p3 259:3 0 97.8G 0 part
├─nvme0n1p4 259:4 0 1000M 0 part
└─nvme0n1p5 259:5 0 20.1G 0 part
$ sudo pvs; vgs; lvs
Error reading device /dev/sda4 at 0 length 4096.
WARNING: Running as a non-root user. Functionality may be unavailable.
/run/lock/lvm/P_global:aux: open failed: Permission denied
WARNING: Running as a non-root user. Functionality may be unavailable.
/run/lock/lvm/P_global:aux: open failed: Permission denied
I am able to boot into Windows, whose D: drive lies on the same HDD with the unencrypted sda4 partition. While Windows warned me about some kind of disk failure surrounding D: drive, I was still able to access and backup all my files present in it safely. I was expecting to be able to do the same thing with my ext4 Linux files, but can not find a way to read them.
Related
STM32 USB Host + HID Joystick
A USB HID Joystick is connected to my STM32F401-based board. I used last versions CubeMX, the USB_HOST library, and HAL. Everything works automatically with the mouse and keyboard, but not with the joystick. I made the following changes in the library based on USBlyzer data:
usbh_conf.h
/*---------- -----------*/
#define USBH_MAX_NUM_ENDPOINTS 1U
/*---------- -----------*/
#define USBH_MAX_NUM_INTERFACES 1U
usbh_core.c
I commented out all sections of the code with the creation of the IN pipe
phost->gState = HOST_ENUMERATION;
phost->Control.pipe_out = USBH_AllocPipe(phost, 0x00U);
// phost->Control.pipe_in = USBH_AllocPipe(phost, 0x80U);
/* Open Control pipes */
// USBH_OpenPipe(phost, phost->Control.pipe_in, 0x80U,
// phost->device.address, phost->device.speed,
// USBH_EP_CONTROL, (uint16_t)phost->Control.pipe_size);
/* Open Control pipes */
USBH_OpenPipe(phost, phost->Control.pipe_out, 0x00U,
phost->device.address, phost->device.speed,
USBH_EP_CONTROL, (uint16_t)phost->Control.pipe_size);
usbh_hid.h
Add nonboot code for my joystick
/* Interface Descriptor field values for HID Boot Protocol */
#define HID_BOOT_CODE 0x01U
#define HID_NONE_BOOT_CODE 0x00U
#define HID_KEYBRD_BOOT_CODE 0x01U
#define HID_MOUSE_BOOT_CODE 0x02U
usbh_hid.c
Add
else if (phost->device.CfgDesc.Itf_Desc[interface].bInterfaceProtocol == HID_NONE_BOOT_CODE)
{
USBH_UsrLog("Non boot device found!");
}
Changed the protocol to non-boot
case HID_REQ_SET_PROTOCOL:
/* set protocol */
classReqStatus = USBH_HID_SetProtocol(phost, 1U);
if (classReqStatus == USBH_OK)
{
HID_Handle->ctl_state = HID_REQ_IDLE;
My code goes as far as getting the USB_HID_GetHIDReportDescriptor in the USB_HID_ClassRequest function and gets it. But then it breaks down on the USB_HID_GetReport function in USB_HID_Process, where I always get the status USB_NOT_SUPPORTED. I need to get a report, my functions will process the data, but I can't get a report and really don't understand why.
Therefore, I need your help, maybe someone has already worked with similar HID devices and USB_HOST library.
Data from USBlyzer:
Connection Status Device connected
Current Configuration 1
Speed Full (12 Mbit/s)
Device Address 6
Number Of Open Pipes 1
Device Descriptor VKB-Sim Space Gunfighter L
Offset Field Size Value Description
0 bLength 1 12h
1 bDescriptorType 1 01h Device
2 bcdUSB 2 0200h USB Spec 2.0
4 bDeviceClass 1 00h Class info in Ifc Descriptors
5 bDeviceSubClass 1 00h
6 bDeviceProtocol 1 00h
7 bMaxPacketSize0 1 08h 8 bytes
8 idVendor 2 231Dh
10 idProduct 2 0127h
12 bcdDevice 2 1997h 19.97
14 iManufacturer 1 01h ". Alex Oz 2012-2020"
15 iProduct 1 02h " VKB-Sim Space Gunfighter L "
16 iSerialNumber 1 00h
17 bNumConfigurations 1 01h
Configuration Descriptor 1 Bus Powered, 500 mA
Offset Field Size Value Description
0 bLength 1 09h
1 bDescriptorType 1 02h Configuration
2 wTotalLength 2 0022h
4 bNumInterfaces 1 01h
5 bConfigurationValue 1 01h
6 iConfiguration 1 00h
7 bmAttributes 1 80h Bus Powered
4..0: Reserved ...00000
5: Remote Wakeup ..0..... No
6: Self Powered .0...... No, Bus Powered
7: Reserved (set to one)
(bus-powered for 1.0) 1.......
8 bMaxPower 1 FAh 500 mA
Interface Descriptor 0/0 HID, 1 Endpoint
Offset Field Size Value Description
0 bLength 1 09h
1 bDescriptorType 1 04h Interface
2 bInterfaceNumber 1 00h
3 bAlternateSetting 1 00h
4 bNumEndpoints 1 01h
5 bInterfaceClass 1 03h HID
6 bInterfaceSubClass 1 00h
7 bInterfaceProtocol 1 00h
8 iInterface 1 00h
HID Descriptor
Offset Field Size Value Description
0 bLength 1 09h
1 bDescriptorType 1 21h HID
2 bcdHID 2 0111h 1.11
4 bCountryCode 1 00h
5 bNumDescriptors 1 01h
6 bDescriptorType 1 22h Report
7 wDescriptorLength 2 012Dh 301 bytes
Endpoint Descriptor 81 1 In, Interrupt, 1 ms
Offset Field Size Value Description
0 bLength 1 07h
1 bDescriptorType 1 05h Endpoint
2 bEndpointAddress 1 81h 1 In
3 bmAttributes 1 03h Interrupt
1..0: Transfer Type ......11 Interrupt
7..2: Reserved 000000..
4 wMaxPacketSize 2 0040h 64 bytes
6 bInterval 1 01h 1 ms
Interface 0 HID Report Descriptor Joystick
Item Tag (Value) Raw Data
Usage Page (Generic Desktop) 05 01
Usage (Joystick) 09 04
Collection (Application) A1 01
Usage Page (Generic Desktop) 05 01
Report ID (1) 85 01
Usage Page (Generic Desktop) 05 01
Usage (X) 09 30
Report Size (16) 75 10
Report Count (1) 95 01
Logical Minimum (0) 15 00
Logical Maximum (4095) 26 FF 0F
Physical Maximum (4095) 46 FF 0F
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Generic Desktop) 05 01
Usage (Y) 09 31
Report Size (16) 75 10
Report Count (1) 95 01
Logical Minimum (0) 15 00
Logical Maximum (4095) 26 FF 0F
Physical Maximum (4095) 46 FF 0F
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Generic Desktop) 05 01
Usage (Rx) 09 33
Report Size (16) 75 10
Report Count (1) 95 01
Logical Minimum (0) 15 00
Logical Maximum (2047) 26 FF 07
Physical Maximum (2047) 46 FF 07
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Generic Desktop) 05 01
Usage (Ry) 09 34
Report Size (16) 75 10
Report Count (1) 95 01
Logical Minimum (0) 15 00
Logical Maximum (2047) 26 FF 07
Physical Maximum (2047) 46 FF 07
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Generic Desktop) 05 01
Usage (Rz) 09 35
Report Size (16) 75 10
Report Count (1) 95 01
Logical Minimum (0) 15 00
Logical Maximum (2047) 26 FF 07
Physical Maximum (2047) 46 FF 07
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Undefined) 05 00
Usage 09 00
Report Size (16) 75 10
Report Count (1) 95 01
Logical Minimum (0) 15 00
Logical Maximum (2047) 26 FF 07
Physical Maximum (2047) 46 FF 07
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Generic Desktop) 05 01
Usage (Z) 09 32
Report Size (16) 75 10
Report Count (1) 95 01
Logical Minimum (0) 15 00
Logical Maximum (2047) 26 FF 07
Physical Maximum (2047) 46 FF 07
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Undefined) 05 00
Usage 09 00
Report Size (16) 75 10
Report Count (1) 95 01
Logical Minimum (0) 15 00
Logical Maximum (2047) 26 FF 07
Physical Maximum (2047) 46 FF 07
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Button) 05 09
Usage Minimum (Button 1) 19 01
Usage Maximum (Button 128) 2A 80 00
Logical Minimum (0) 15 00
Logical Maximum (1) 25 01
Report Size (1) 75 01
Report Count (128) 96 80 00
Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit) 81 02
Usage Page (Generic Desktop) 05 01
Usage (Hat Switch) 09 39
Logical Minimum (0) 15 00
Logical Maximum (7) 26 07 00
Physical Minimum (0) 35 00
Physical Maximum (360) 46 68 01
Unit (Eng Rot: Degree) 65 14
Unit Exponent (1) 55 01
Report Size (4) 75 04
Report Count (1) 95 01
Input (Data,Var,Abs,NWrp,Lin,Pref,Null,Bit) 81 42
Usage (Undefined) 09 00
Unit (None) 65 00
Unit Exponent (0) 55 00
Report Size (4) 75 04
Report Count (3) 95 03
Input (Cnst,Ary,Abs) 81 01
Usage Page (Generic Desktop) 05 01
Usage (Undefined) 09 00
Report Size (16) 75 10
Report Count (1) 95 01
Input (Cnst,Ary,Abs) 81 01
Usage Page (Generic Desktop) 05 01
Usage (Undefined) 09 00
Report Size (16) 75 10
Report Count (1) 95 01
Input (Cnst,Ary,Abs) 81 01
Usage Page (Generic Desktop) 05 01
Usage (Undefined) 09 00
Report Size (16) 75 10
Report Count (1) 95 01
Input (Cnst,Ary,Abs) 81 01
Usage Page (Generic Desktop) 05 01
Usage (Undefined) 09 00
Report Size (8) 75 08
Report Count (23) 95 17
Input (Cnst,Ary,Abs) 81 01
Report ID (11) 85 0B
Usage Page (Generic Desktop) 05 01
Usage (Undefined) 09 00
Report Size (8) 75 08
Report Count (63) 95 3F
Input (Cnst,Ary,Abs) 81 01
Report ID (8) 85 08
Usage Page (Generic Desktop) 05 01
Usage (Undefined) 09 00
Report Size (8) 75 08
Report Count (63) 95 3F
Input (Cnst,Ary,Abs) 81 01
Logical Minimum (0) 15 00
Logical Maximum (255) 26 FF 00
Physical Maximum (255) 46 FF 00
Report ID (88) 85 58
Report Size (8) 75 08
Report Count (63) 95 3F
Usage (Undefined) 09 00
Output (Data,Var,Abs,NWrp,Lin,Pref,NNul,NVol,Bit) 91 02
Report ID (89) 85 59
Report Size (8) 75 08
Report Count (128) 95 80
Usage (Undefined) 09 00
Feature (Data,Var,Abs,NWrp,Lin,Pref,NNul,NVol,Bit) B1 02
End Collection C0
U-boot: how to check if tftp command successfully loaded image into ram?
I load a rootfs image into RAM via u-boot tftp and flash it to the device flash storage. This is currently done manually, but now I want to do it automatically via a u-boot script:
tftp ${rootfs_image};
mmc write ${loadaddr} ${blk} ${cnt}
However, when it looks for an image from the tftp server with the u-boot tftp ${rootfs_image} command and it DOESN'T find the image, I don't want to run the mmc write part of the script.
How do I check if the tftp command successfully downloaded the image into RAM?
Using the TFTP protocol does not ensure that the integrity of the transferred data will be preserved - see section Security Consideration in this article. Assuming your u-boot has the hash command available, or that you can re-compile it with CONFIG_CMD_HASH=y, you could use an SHA-256 hash for verifying that your image was properly transferred: On a Linux TFTP server: # create an image for the purpose of the example echo "Binary Image" > image.bin # display sha256 hash for image.bin sha256sum -b image.bin 36949f85f1bff0d5d1dd5fcfdfd725e919b0ee64be24f7f3ccfb53908fd09550 *image.bin # create a file containing the hash in binary # credits: sha256sum -b image.bin | xxd -r -p > image.bin.sha256sum.bin # display content of binary file hexdump -C image.bin.sha256sum.bin 00000000 36 94 9f 85 f1 bf f0 d5 d1 dd 5f cf df d7 25 e9 |6........._...%.| 00000010 19 b0 ee 64 be 24 f7 f3 cc fb 53 90 8f d0 95 50 |...d.$....S....P| 00000020 On your u-boot system (using the memory layout available on my Alwinner H5 system here): # 0x40080000: address where image.bin will be transfered # 0x40090000: address where image.bin.sha256sum.bin will be transfrered # 0x40090000: address where the sha256 has will be computed by u-boot on the 13 bytes of image.bin # clearing memory mw.b 0x40080000 0 0x2000 mw.b 0x40090000 0 0x20 mw.b 0x400A0000 0 0x20 md.b 0x40080000 0x20 40080000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 40080010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ md.b 0x40090000 0x20 40090000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 40090010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ md.b 0x400A0000 0x20 400a0000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 400a0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ tftp 0x40080000 image.bin Using ethernet#1c30000 device TFTP from server 192.168.1.22; our IP address is 192.168.1.2 Filename 'image.bin'. Load address: 0x40080000 Loading: # 5.9 KiB/s done Bytes transferred = 13 (d hex) tftp 0x40090000 image.bin.sha256sum.bin Using ethernet#1c30000 device TFTP from server 192.168.1.22; our IP address is 192.168.1.2 Filename 'image.bin.sha256sum.bin'. Load address: 0x40090000 Loading: # 15.6 KiB/s done Bytes transferred = 32 (20 hex) md.b 0x40090000 0x20 40090000: 36 94 9f 85 f1 bf f0 d5 d1 dd 5f cf df d7 25 e9 6........._...%. 40090010: 19 b0 ee 64 be 24 f7 f3 cc fb 53 90 8f d0 95 50 ...d.$....S....P hash sha256 0x40080000 0x0d *0x400A0000 sha256 for 40080000 ... 4008000c ==> 36949f85f1bff0d5d1dd5fcfdfd725e919b0ee64be24f7f3ccfb53908fd09550 cmp.b 0x40090000 0x400A0000 0x20 Total of 32 byte(s) were the same echo $? 0 In the case image.bin and/or image.bin.sha256sum.bin would have been improperly transferred, the chances that the computed sha256 would match the transferred one are extremely unlikely - using SHA-512 would make this even more unlikely. The outcome would have been in the case of an incorrect transfer: echo $? 1 In real life, this would be more practical to transfer an image with a fixed, maximum length, for example padded with zeroes, so that a u-boot script responsible for validating the transferred image would use a fixed length, say 8 KiB, that is 0x2000 bytes. ls -lgG image.bin -rw-rw-r-- 1 13 Dec 17 20:34 image.bin dd if=/dev/zero of=image.bin bs=8K count=1 oflag=append ls -lgG image.bin -rw-rw-r-- 1 8192 Dec 17 21:03 image.bin hexdump -C image.bin 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00002000 The correct u-boot command to use for computing the hash would be: hash sha256 0x40080000 0x2000 *0x400A0000 And a new binary file containing the new hash would of course have to be created as well: sha256sum -b image.bin | xxd -r -p > image.bin.sha256sum.bin I used two files for the purpose of the example, but you could just append image.bin.sha256sum.bin to image.bin and transfer one single file. You would have to replace 0x400A0000 by 0x40082000 in the hash and cmp commands. I hope this helps.
The tftp command returns true if it succeeds. So you could write:
tftp ${rootfs_image} && mmc write ${loadaddr} ${blk} ${cnt}
Now mmc write will only be executed if the tftp command succeeds.
h264 inside AVI, MP4 and "Raw" h264 streams. Different format of NAL units (or ffmpeg bug)
TL;DR: I want to read raw h264 streams from AVI/MP4 files, even broken/incomplete. Almost every document about h264 tells me that it consists of NAL packets. Okay. Almost everywhere told to me that the packet should start with a signature like 00 00 01 or 00 00 00 01. For example, https://stackoverflow.com/a/18638298/8167678, https://stackoverflow.com/a/17625537/8167678 The format of H.264 is that it’s made up of NAL Units, each starting with a start prefix of three bytes with the values 0x00, 0x00, 0x01 and each unit has a different type depending on the value of the 4th byte right after these 3 starting bytes. One NAL Unit IS NOT one frame in the video, each frame is made up of a number of NAL Units. Okay. I downloaded random_youtube_video.mp4 and strip out one frame from it: ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.avi And got: Red part - this is part of AVI container, other - actual data. As you can see, here I have 00 00 24 A9 instead of 00 00 00 01 This AVI file plays perfectly I do same for mp4 container: As you can see, here exact same bytes. This MP4 file plays perfectly I try to strip out raw data: ffmpeg -i pic.avi -c copy pic.h264 This file can't play in VLC or even ffmpeg, which produced this file, can't parse it: I downloaded mp4 stream analyzer and got: MP4Box tells me: Cannot find H264 start code Error importing pic.h264: BitStream Not Compliant It very hard to learn internals of h264, when nothing works. So, I have questions: What actual data inside mp4? What I must read to decode that data (I mean different annex-es) How to read stream and get decoded image (even with ffmpeg) from this "broken" raw stream? UPDATE: It seems bug in ffmpeg: When I do double conversion: ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.mp4 ffmpeg pic.mp4 -c copy pic.h264 But when I convert file directly: ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.h264 I have NALs signatures and one extra NAL unit. Other bytes are same (selected). This is bug? UPDATE Not, this is not bug, U must use option -bsf h264_mp4toannexb to save stream as "Annex B" format (with prefixes)
"I want to read raw h264 streams from AVI files, even broken/incomplete."
"Almost everywhere told to me that the packet should start with a signature like : 00 00 01 or 00 00 00 01"
"...As you can see, here I have 00 00 24 A9 instead of 00 00 00 01"
Your H264 is in AVCC format which means it uses data sizes (instead of data start codes). It is only Annex-B that will have your mentioned signature as start code.
You seek frames, not by looking for start codes, but instead you just do skipping by frame sizes to reach the final correct offset of a (requested) frame...
AVI processing :
Read size (four) bytes (32-bit integer, Little Endian).
Extract the next following bytes up to size amount.
This is your H.264 frame (in AVCC format), decode the bytes to view image.
To convert into Annex-B, try replacing first 4 bytes of H.264 frame bytes with 00 00 00 01.
Consider your shown AVI bytes (see first picture) :
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 4C 49 53 54 BA 24 00 00 6D 6F 76 69 ....LISTº$..movi
30 30 64 63 AD 24 00 00 00 00 24 A9 65 88 84 27 00dc.$....$©eˆ„'
C7 11 FE B3 C7 83 08 00 08 2A 7B 6E 59 B5 71 E1 Ç.þ³Çƒ...*{nYµqá
E3 9C 0E 73 E7 10 50 00 18 E9 25 F7 AA 7D 9C 30 ãœ.sç.P..é%÷ª}œ0
E6 2F 0F 20 00 3A 64 AA CA 5E 4F CA FF AE 20 04 æ/. .:dªÊ^OÊÿ® .
07 81 40 00 48 00 0A 28 71 21 84 48 06 18 90 0C ..#.H..(q!„H....
31 14 57 9E 7A CD 63 A0 E0 9B 96 69 C5 18 AE F2 1.WžzÍc à›–iÅ.®ò
E6 07 02 29 01 20 10 70 A1 0F 8C BC 73 F0 78 FA æ..). .p¡.Œ¼sðxú
9E 1D E1 C2 BF 8C 62 CE CE AC 14 5A A4 E1 45 44 ž.á¿ŒbÎά.Z¤áED
38 38 85 DB 12 57 3E F6 E0 FB AE 03 04 21 62 8D 88…Û.W>öàû®..!b.
F6 F1 1E 37 1C A2 FF 75 1C F1 02 66 0C 92 07 06 öñ.7.¢ÿu.ñ.f.’..
15 7C 90 15 6F 7D FC BD 13 1E 2B 0C 14 3C 0C 00 .|..o}ü½..+..<..
B0 EA 6F 53 B4 98 D7 80 7A 68 3E 34 69 20 D2 FA °êoS´˜×€zh>4i Òú
F0 91 FC 75 C6 00 01 18 C0 00 3B 9A C5 E2 7D BF ð‘üuÆ...À.;šÅâ}¿
Some explanation :
Ignore leading multiple 00 bytes.
4C 49 53 54 D6 3C 00 00 6D 6F 76 69 including 30 30 64 63 = AVI "List" header.
AD 24 00 00 == decimal 9389 is AVI's own size of H264 item (must read in Little Endian).
Notice that the AVI bytes include...
- a note of item's total size (AD 24 00 00... or reverse for Little Endian : 00 00 24 AD)
- followed by item data (00 00 24 A9 65 88 84 27 ... etc ... C5 E2 7D BF).
This size includes both the 4 bytes of the AVI's"size" entry + expected bytes length of the item's own bytes. Can be written simply as:
AVI_Item_Size = ( 4 + item_H264_Frame.length );
H.264 video frame bytes in AVI :
Next follows the item data, which is the H.264 video frame. By sheer coincidence of formats/bytes layout, it too holds a 4-byte entry for data's size (since your H264 is in AVCC format, if it was Annex-B then you would be seeing start code bytes here instead of size bytes).
Unlike AVI bytes, these H264 size bytes are written in Big Endian format.
00 00 24 A9 = size of bytes for this video frame (instead of start code : 00 00 00 01).
65 88 84 27 C7 11 FE B3 C7 = H.264 keyframe (always begins X5, where the X value is based on other settings).
Remember after four size bytes (or even start codes) if followed by...
byte X5 = keyframe (IDR), example byte 65.
byte X1 = P or B frame, example byte 41.
byte X6 = SEI (Supplemental Enhancement Information).
byte X7 = SPS (Sequence Parameter Set).
byte X8 = PPS (Picture Parameter Set).
bytes 00 00 00 X9 = Access unit delimiter.
You can find the H.264 if you search for exact same bytes within AVI file. See third picture, these are your H.264 bytes (they are cut & pasted into the AVI container).
Sometimes a frame is sliced into different NAL units. So if you extract a key frame and it only shows 1/2 or 1/3 instead of full image, just grab next one or two NAL and re-try the decode.
Delete an index with double quotes
I had a problem with an index. The index name is "dbo.indexname" (including '"') and I cannot delete it. Someone knows how to drop an index with that name? sp_help table output: index_name, index_keys, index_description, index_max_rows_per_page, index_fillfactor, index_reservepagegap, index_created, index_local "dbo.index_name", id, nonclustered, 0, 0, 0, Nov 27 2015 6:41PM, Global Index Output for: select * from sysindexes where name like '%dbo.index_name%' name id indid doampg ioampg oampgtrips status3 status2 ipgtrips first root distribution usagecnt segment status maxrowsperpage minlen maxlen maxirow keycnt keys1 "dbo.index_name" 1259148500 3 0 0 0 0 0 0 0 0 0 0 1 0 0 15 15 15 2 81 00 06 00 01 00 00 bf 08 00 00 00 01 00 00 00 81 00 00 00 00 00 00 2d 06 00 00 00 09 00 00 00
What is the name of the table? Let's assume 'mytab'. Now do this: set quoted_identifier on go drop index mytab."dbo.indexname" go For some background on why this is puzzling, see http://www.sypron.nl/quiz2002a.html#dec02 .
DNS Query Structure
When I am sending a DNS query to the DNS it returns the header with the format bit set. Indicating there is a problem with the format, but I am failing to see what it is. Its possible I have misinterpreted the RFC, or misread it but right now I cant seem to work it out. The DNS structure I am sending looks like this in hex. Header 00 01 - ID = 1 01 00 - RD = 1 00 01 - QD = 1 00 00 - AN 00 00 - NS 00 00 - NR Question for www.google.com 03 77 - 3 w 77 77 - w w 06 67 - 6 g 6f 6f - o o 67 6c - g l 65 03 - e 3 63 6f - c o 6d 00 - m 0 00 01 - QTYPE 00 01 - QCLASS I then flip the bytes for any field that is two bytes, to convert to big endian for the network format. So each row of the header, and then QTYPE and QCLASS ...
Here's what a byte-by-byte hexdump of that query packet should look like (tested and working!): 00000000 00 01 01 00 00 01 00 00 00 00 00 00 03 77 77 77 |.............www| 00000010 06 67 6f 6f 67 6c 65 03 63 6f 6d 00 00 01 00 01 |.google.com.....| I think your problem is that the third and fourth bytes of the packet (flags and rcode) are two single-byte fields, not one 2-byte field - it looks like you might be treating it as a 16 bit integer and swapping the bytes?
To get these you can use netcat and dig. # nc –uip 53 > dnsreqdump # dig www.example.com #localhost # nc –u 8.8.8.8 53 <dnsreqdump >dnsrespdump Now you can inspect them in hexedit or your favorite hex editor.
I tend to think that your problem depends on how are you actually "flipping the bits to convert to network format". Typical C library implementations provide the htonl() function family to do the conversion from host into network order and viceversa. Of course, without seeing the code, I cannot be sure that this is the problem.