I run a server in ESP8266 which accept argument and value when you click the submit button in the webpage. after some iteration of clicking the submit button, the esp8266 crashes with the Exception of:
--------------- CUT HERE FOR EXCEPTION DECODER ---------------
Exception (29):
epc1=0x40205224 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000
>>>stack>>>
ctx: cont
sp: 3ffffca0 end: 3fffffc0 offset: 0190
3ffffe30: 3fff1598 3fff1598 3ffffe80 4020521f
3ffffe40: 3fff6354 3ffffe98 3ffffe80 40217ef0
3ffffe50: 00000003 3ffffec0 3ffffe80 40217f20
3ffffe60: 3ffffe98 3ffffea4 3fff1588 3fff0da4
3ffffe70: 00000000 3ffffec0 00000005 402053a9
3ffffe80: 3fff631c 0029002f 00003a30 3fff6300
3ffffe90: 0029002f 800001da 3fff6354 0029002f
3ffffea0: 00000001 3fffbf5c 0005002f 00217d42
3ffffeb0: 00000001 3fff0dac 3ffffefc 4020cb24
3ffffec0: 3fff68b4 01da01df 00000000 fffffffe
3ffffed0: 00000000 3fffc6fc 00000000 3fff1c88
3ffffee0: 00000000 3fffdad0 40223278 00000000
3ffffef0: 3ffffefc 3fffff64 3fff0a00 40223428
3fffff00: 00000000 000003e8 3fffff60 03e87701
3fffff10: 402583d1 00000004 3fff0d58 4020171a
3fffff20: 40258677 00000004 3fff0d58 3fff0d5c
3fffff30: 00000000 4bc6a7f0 0189374b 00000000
3fffff40: 00000000 00000000 4bc6a7f0 00000000
3fffff50: 000e001a 00050008 4010051c 0001fc51
3fffff60: 00000000 6c80e048 4024c800 3ffe8514
3fffff70: 024bbda9 80000000 00000000 4010059d
3fffff80: 00000000 00000000 00000001 3fff1cc8
3fffff90: 3fffdad0 00000000 0001fc51 40211f35
3fffffa0: 3fffdad0 00000000 3fff1c88 40218fd4
3fffffb0: feefeffe feefeffe 3ffe899c 40100f45
<<<stack<<<
last failed alloc call: 4020521F(100)
--------------- CUT HERE FOR EXCEPTION DECODER ---------------
ets Jan 8 2013,rst cause:2, boot mode:(3,6)
For debugging: Free Heap is printed(runs every 2 seconds using millis) in the serial monitor by
Serial.println(ESP.getFreeHeap());
The submit button is sending the following argument and value(Partial only) :
http://192.168.4.1?user=1234&password=1234&data1=qwertyuiopasdfghjklzxcvbnm&data1=qwertyuiopasdfghjklzxcvbnm&data2=qwertyuiopasdfghjklzxcvbnm&data3=qwertyuiopasdfghjklzxcvbnm&data4=qwertyuiopasdfghjklzxcvbnm
as you can see the FreeHeap is going down until it reaches the minimum where ESP8266 can't handle.
Is there a way to refresh or reuse the heap without rebooting the Chip?
Is safeString can help to this problem ?
in ruby.h, I find it difficult to understand this macro:
#define SYMBOL_P(x) (((VALUE)(x)&0xff)==SYMBOL_FLAG)
I have no idea why this & operation is executed with 0xff. Doesn't every number & 0xff equal itself?
& is a bitwize operator (AND), (remember logic table?)
0 & 0 = 0
1 & 0 = 0
0 & 1 = 0
1 & 1 = 1
so what it does here?
0xff is the hexa of 255
in binary (DWORD): 00000000 00000000 00000000 11111111
so assuming a number x= any_value
the representation of x can be like this
???????? ???????? ???????? ????????
each ? can be either 1 or 0
so applying bitwize operator & (AND) with the mask 0xff gives
???????? ???????? ???????? ????????
&
00000000 00000000 00000000 11111111
=
00000000 00000000 00000000 ????????
for example
00000000 00000000 00000011 00000011
&
00000000 00000000 00000000 11111111
=
00000000 00000000 00000000 00000011
^________________________^ ^______^
zeroed kept
The VALUE type in the Ruby source code is usually 32-bit or 64-bit, so the & 0xFF sets all the bits except the lowest 8 to 0.
EDIT
Indeed, there is no direct way of editing the range values in memory.
Thanks #AndASM for the detailed answer and Carl; good hunch, it was spot-on. I must have been too tangled in all the reversing at that moment, forgetting Value2 is just a property.
Meanwhile, I delved a bit deeper with some other tests and debugging with OllyDbg and found a few interesting things:
Cells are arranged in 16 x 1024 areas. The structure holding the
areas may very well be the worksheet, but I can't confirm yet;
Each time the Value property is invoked, absolute sheet
offsets (row, column) are used to find the corresponding area and
then for some indexing inside the area to get the actual value;
A 2D SAFEARRAY of type VARIANT is created;
Values are not retrieved in a contiguous block, but individually.
This means that each "point" in a range (row,col) is sent to the
indexing procedure to return a value (variant, obviously) for its
corresponding SAFEARRAY element;
As a consequence of the above, each time you retrieve a value via
Range.Value2(row,col), the whole process is reiterated for all the
values in the range. Imagine the performance hit if you do this
multiple times inside a procedure or, even worse, inside a loop.
Just don't; you're better off creating a copy of Value2 and
addressing it via indexing;
Last, yet not least, the distribution of values inside
SAFEARRAY.pvData is column-based (col,row), not row-based, which
may be found counter-intuitive and at odds with the VBA indexing
mode, which is (row,col). This may come in handy if you need
accessing pvData directly in memory and keeping dimension coherence.
As an example, a range like the one below
1, 2, 3, 4
5, 6, 7, 8
would be stored in pvData in the following order:
1, 5, 2, 6, 3, 7, 4, 8
I hope the above helps.
To sum up, in absence of any such exported function in Excel, the best way around is creating a copy of Value2, sorting / manipulating it towards the desired outcome and assigning it back to the range property.
I have recently finished a variation of QuickSort and intend to implement in for Excel. The algorithm is effective and would really bring value as an add-in, if not for the extra time spent on putting the array values into the range. Transposing only works for less than 65537, while the 'paste variant-array to range takes too long on large sorts.
So, I wrote a few procedures that would allow copying the 2D values from the range into a 1D array (1D is needed for sorting) and (after the sorting is done) putting them back, all based on SAFEARRAY and MemCopy(RtlMoveMemory) and, alternately, WriteProcessMemory.
All works well, as far as memory operations are concerned:
- the range values are copied to the array (from one SafeArray.pvData to the other);
- the array values (after running the sorting algo) are successfully copied to the Range.Value2 SafeArray.pvData.
Still, the range does not update, as it seems to flip back to the old values (more on that in the code below).
Why would "Range.Value2 = SomeOther2dArray" work and not modifying the data directly in memory? I have a feeling I am missing something here. Is a formula sort / update needed, as well?
Here is the main procedure:
Public Sub XLSORT_Array2()
With Application
screenUpdateState = .ScreenUpdating
statusBarState = .DisplayStatusBar
calcState = .Calculation
eventsState = .EnableEvents
.ScreenUpdating = False
.DisplayStatusBar = False
.Calculation = xlCalculationManual
.EnableEvents = False
End With
Dim rngSort As Range
Dim arrSort() As Variant
Dim arrTemp As Variant
Dim i As Long
Dim dblTime As Double
Dim dblInitTime As Double: dblInitTime = Timer
Set rngSort = Selection
If Not rngSort Is Nothing Then
If rngSort.Cells.Count > 1 And rngSort.Areas.Count = 1 Then
dblTime = Timer
ReDim arrSort(1 To rngSort.Cells.Count)
Debug.Print Timer - dblTime & vbTab & "(Redim)"
'just testing Excel memory location
'Debug.Print VarPtr(rngSort.Value2(1, 1))
dblTime = Timer
SA_Duplicate arrSort, rngSort.Value2
Debug.Print Timer - dblTime & vbTab & "(Copy)"
dblTime = Timer
SORTVAR_QSWrapper arrSort, 1, rngSort.Cells.Count
Debug.Print Timer - dblTime & vbTab & "(Sort)"
'this would be the fastest method
'variants are copied to memory
'yet the range does not update with the new values
SA_Duplicate rngSort.Value2, arrSort
'dblTime = Timer
'looping = too slow
'For i = 1 To rngSort.Cells.Count
' rngSort.Cells(i).Value = arrSort(i)
'Next
'this works, but it's too slow, as well
'If rngSort.Cells.Count > 65536 Then
' ReDim arrTemp(LBound(rngSort.Value2, 1) To UBound(rngSort.Value2, 1), LBound(rngSort.Value2, 2) To UBound(rngSort.Value2, 2))
' SA_Duplicate arrTemp, arrSort
' rngSort.Value2 = arrTemp
'Else
' rngSort.Value2 = WorksheetFunction.Transpose(arrSort)
' Debug.Print "Transposed"
'End If
'Debug.Print Timer - dblTime & vbTab & "(Paste)"
End If
End If
With Application
.ScreenUpdating = screenUpdateState
.DisplayStatusBar = statusBarState
.Calculation = calcState
.EnableEvents = eventsState
End With
Debug.Print VarPtr(rngSort.Value2(1, 1)) & vbTab & Mem_ReadHex(ByVal VarPtr(rngSort.Value2(1, 1)), rngSort.Cells.Count * 16)
Set rngSort = Nothing
Debug.Print Timer - dblInitTime & vbTab & "(Total Time)" & vbNewLine
End Sub
Let's say the values in the range are 4, 3, 2, and 1.
Before SA_Duplicate arrSort, rngSort.Value2 the memory reads:
130836704 05000000 00000000 00000000 00001040 05000000 00000000 00000000 00000840 05000000 00000000 00000000 00000040 05000000 00000000 00000000 0000F03F
129997032 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
where 130836704 is Range.Value2 SafeArray.pvData and 129997032 is SortArray SafeArray.pvData. Each 16-byte batch represents the variant actual data, as read from memory (no LE translation, just in hex), with the first 2 bytes indicating the VarType. In this case, vbDouble.
After the copy, as expected, the memory reads:
130836704 05000000 00000000 00000000 00001040 05000000 00000000 00000000 00000840 05000000 00000000 00000000 00000040 05000000 00000000 00000000 0000F03F
129997032 05000000 00000000 00000000 00001040 05000000 00000000 00000000 00000840 05000000 00000000 00000000 00000040 05000000 00000000 00000000 0000F03F
After the sort is complete, the SortArray SafeArray.pvData reads:
129997032 05000000 00000000 00000000 0000F03F 05000000 00000000 00000000 00000040 05000000 00000000 00000000 00000840 05000000 00000000 00000000 00001040
After executing SA_Duplicate rngSort.Value2, arrSort, the memory shows that the Range.Value2 SafeArray.pvData has been updated:
129997032 05000000 00000000 00000000 0000F03F 05000000 00000000 00000000 00000040 05000000 00000000 00000000 00000840 05000000 00000000 00000000 00001040
130836704 05000000 00000000 00000000 0000F03F 05000000 00000000 00000000 00000040 05000000 00000000 00000000 00000840 05000000 00000000 00000000 00001040
All looking fine so far, except that Debug.Print VarPtr(rngSort.Value2(1, 1)) & vbTab & Mem_ReadHex[...] shows that the values flipped back to the initial order:
130836704 05000000 00000000 00000000 00001040 05000000 00000000 00000000 00000840 05000000 00000000 00000000 00000040 05000000 00000000 00000000 0000F03F
Please share any thoughts or methods you found effective. Any help is appreciated. It's frustrating having to wait for Excel about 4 seconds ( sorting 1,000,000 + cells), when even the most challenging sort takes less than that.
Thanks in advance!
Well, you didn't provide a working implementation of several vital parts, especially SA_Duplicate, so this is mostly guess work. But, I think the answer is likely quite simple.
Range.Value2 is a property, not a variable. So behind the scenes it is actually two functions, lets call them Range.let_Value2 and Range.get_Value2.
That said, how do you expect the call SA_Duplicate rngSort.Value2, arrSort to work? Because what I see is SA_Duplicate rngSort.get_Value2, arrSort. I would assume that rngSort.get_Value2 is creating a new SafeArray, then copying the data from Excel's internal data structures into that SafeArray. If I'm correct, you're then writing your data to a temporary buffer that VBA later discards and Excel has already forgotten about.
You need to use rngSort.let_Value2 arrSort which is usually called as rngSort.Value2 = arrSort.
As a side note, if get_Value2 allocates a new array as I think it does, both the SA_Duplicate calls are unnecessary and you might be able to sort the returned array in place. Do remember to pass it to let_Value2 by assigning the array variable back to the property when you are done.
I saw a line in xen's kernel code (file: xen/include/asm-x86/x86_64/page.h), but cannot understand why they are doing this:
/* Extract flags into 24-bit integer, or turn 24-bit flags into a pte mask. */
#define get_pte_flags(x) (((int)((x) >> 40) & ~0xFFF) | ((int)(x) & 0xFFF))
#define put_pte_flags(x) (((intpte_t)((x) & ~0xFFF) << 40) | ((x) & 0xFFF))
As to
#define get_pte_flags(x) (((int)((x) >> 40) & ~0xFFF) | ((int)(x) & 0xFFF))
I understand ((int)(x) & 0xFFF) will extract the last 24 bits of x, but why they need the first part ((int)((x) >> 40) & ~0xFFF) ?
As to
#define put_pte_flags(x) (((intpte_t)((x) & ~0xFFF) << 40) | ((x) & 0xFFF))
I'm lost at the purpose of ((intpte_t)((x) & ~0xFFF) << 40). It should be 0 in my opinion. Then why do we need it?
Thanks,
I had to look twice at their code. Because it took me a minute to realize that 0xFFF is not 24 bits, it's only 12 bits. So take an example 64 bit input: 0xAABBCCDDEEFF1122. Shift it right by 40, and you get 0x0000000000AABBCC. ~0xFFF is shorthand in this case for 0xFFFFFFFFFFFFF000. And them together, and you get 0x0000000000AAB000. So basically, they grabbed the top 12 bits and moved them down. Then they or that with the bottom 12 bits. So they end up with 0x0000000000AAB122.
The other half does the opposite, takes 24 bits at the bottom, cuts them in half and puts 12 at the top and 12 at the bottom.
0xFFF is not 24 one-bits, it's only 12.
Knowing this, you'll see that the purpose of get_pte_flags is to move the top 12 bits into position 12-24, like so:
xxxxxxxx xxxx0000 00000000 00000000 00000000 00000000 0000yyyy yyyyyyyy
becomes
00000000 00000000 00000000 00000000 00000000 xxxxxxxx xxxxyyyy yyyyyyyy
Of course, put_pte_flags does the inverse, moving the bits back to the most significant position.
Think 64 bit.
On a 32 bit system the result would be 0, of course.
But, when you shift 24 bit 40 bits left, you have
xxxxxxxx yyyyyyyy zzzzzzzz 00000000 00000000 00000000 00000000 00000000
which is a valid 64 bit value.
The Deadlock occurs in my application when initialization of local static variable happens in the function called from DLLMain Entry point with param DLL_THREAD_DETACH.
Below is Windbg analysis
This is usually caused by another thread holding the loader lock.
Following are the Locks Held.
CritSec ntdll!LdrpLoaderLock+0 at 7c97e178
LockCount 3
RecursionCount 1
OwningThread 17e8
EntryCount d
ContentionCount d
*** Locked
CritSec MSVCR80!__app_type+94 at 781c3bc8
LockCount 1
RecursionCount 1
OwningThread 1100
EntryCount 1
ContentionCount 1
*** Locked
#
Call stack Thread 17e8
781c3bc8 78132bd9 0777fde4 ntdll!RtlEnterCriticalSection+0x46
00000008 b87d2630 00000000 MSVCR80!_lock+0x2e
0864ae10 08631d7f 0864ae10 EPComUtilities32!_onexit+0x36
0864ae10 b87d2588 00000001 EPComUtilities32!atexit+0x9
0777fea8 0864719f 08630000 EPComUtilities32!XCriticalSectionEx::ThreadTerminated+0x5f
08630000 00000003 00000000 EPComUtilities32!DllMain+0x20
08630000 7c90118a 08630000 EPComUtilities32!__DllMainCRTStartup+0x7a
08630000 00000003 00000000 EPComUtilities32!_DllMainCRTStartup+0x1d
#
Call Stack thread 1100
000000b0 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
000000b0 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xa8
000000b0 ffffffff 06ce64e0 kernel32!WaitForSingleObject+0x12
000480ba 000f4240 00000000 CATSysMultiThreading!CATThreads::Join+0xf5
0012fcc8 00000004 00000000 JS0GROUP!CATLM::StopHB+0xf4
d138509f 00416694 00000001 JS0GROUP!CATLM::Unmake+0x6b
00000000 00000000 00000000 MSVCR80!_cinit+0xd6
00000000 0012fd6c 081e68d9 MSVCR80!exit+0xd
00000000 06d404f0 0998fb90 JS0GROUP!CATExit+0x1d
00000000 004ef366 0000000d DNBPLMProvider!DNBEPLMTransactionMgt::OnApplicationExit+0x229
00000000 0012fd9c 004eabfc JS0GROUP!CATCallExits+0x2bc
00000000 0012ff7c 0040cefd JS0GROUP!CATErrorNormalEnd+0x31
00000000 06ce71d0 06ce71d0 JS0GROUP!CATExit+0xc
00000007 06cdb120 059b61d8 DLMMfgContextSolver!main+0x146d
ffffffff ffffffff bffde000 DLMMfgContextSolver!__tmainCRTStartup+0x10f
#
Please give you comments to understand what might have caused the deadlock.
Note: the moment i make the static variable as non static the problem disappears this in context of example posted in forum Deadlock occurs in Function Scoped Static variables (Thread Unsafe in VC++)
In short, what caused the deadlock is that you did something non-trivial in DllMain.