I am running a DTLS Server which handles more than 500 connections, every time some connections are closed i see there is some RAM memory utilization increases.
I wonder there is a leak in memory from my below approach of Initialize_Sever_Context , create_connexion and close_connexion. The exact code is too big to create actual scenario, so i just outlined the step.
Pls let me know if any extra information is required?
I am using OpenSSL version 1.1.1k on Linux.
//connect_info structure user defined
{
void* sll;
void* bio;
....
}array_of_connections
*connect_info = &array_of_connections;
// global
SSL_CTX* server_ctx;
Initialize_Sever_Context()
{
// server_ctx is global
server_ctx = SSL_CTX_new(DTLS_server_method());
X509_VERIFY_PARAM *local_vpm = X509_VERIFY_PARAM_new()
//setting verify flags, cookie flags and cypher lists etc..
//....
SSL_CTX_set1_param(server_ctx, local_vpm);
}
create_connexion(connect_info)
{
// server_ctx is global
ssl = SSL_new(server_ctx);
bio = BIO_new_dgram(handler, BIO_NOCLOSE);
..
..
SSL_set_bio(ssl, bio, bio);
connect_info->ssl = ssl;
connect_info->bio = bio;
}
handle_closed_connexions()
{
for(conn = 1; conn<MAX_CONN;conn++)
{
close_connexion(connect_info[conn]);
}
}
close_connexion(connect_info)
{
// store prev ssl objects
SLL *local_ssl = connect_info -> ssl;
// make setup ready for the next connexions
// and start listening
create_connexion(connect_info)
// free the previous closed connections
SSL_free(local_ssl);
}
Inside SSL_free we have BIO_free_all(s->rbio), BIO_free_all(s->rbio) and BIO_CTX_free(s->ctx) and finally OPENSSL_free(s)
As far as i understand when we do SSL_free, all the members(pointers) inside SLL object are freed.
But inside OpenSSL non of pointers are set to NULL after free(), so i expect the application to crash.
But my application is working even after the pointers are freed.
Why does not OpenSSL set the pointers to NULL after they are freed or Can i assume that my application is Safe with the above approach?
I have checked posts
1
2 and others, but none of them satisfy my requirement, so i am asking a new question.
Why does not OpenSSL set the pointers to NULL after they are freed or Can i assume that my application is Safe with the above approach?
99.99% of the time, the pointers aren't going to be accessed again, so setting them to NULL serves no purpose. If you do happen to need them set to NULL, then you can set them to NULL.
Related
I am trying to write a remapping target for usage with DM.
I followed instructions from several places (including this Answer) all essentially giving the same code.
This is ok, but not enough for me.
I need to modify "in transit" data of struct bio being remapped.
This means I need to make a deep-clone of the bio, including the data; apparently the provided functions (e.g.: bio_clone_bioset()) do not copy data at all, but point iovec's to the original pages/offsets.
I tried some variations of the following scheme:
void
mt_copy(struct bio *dst, struct bio *src) {
struct bvec_iter src_iter, dst_iter;
struct bio_vec src_bv, dst_bv;
void *src_p, *dst_p;
unsigned bytes;
unsigned salt;
src_iter = src->bi_iter;
dst_iter = dst->bi_iter;
salt = src_iter.bi_sector;
while (1) {
if (!src_iter.bi_size) {
break;
}
if (!dst_iter.bi_size) {
break;
}
src_bv = bio_iter_iovec(src, src_iter);
dst_bv = bio_iter_iovec(dst, dst_iter);
bytes = min(src_bv.bv_len, dst_bv.bv_len);
src_p = kmap_atomic(src_bv.bv_page);
dst_p = kmap_atomic(dst_bv.bv_page);
memcpy(dst_p + dst_bv.bv_offset, src_p + src_bv.bv_offset, bytes);
kunmap_atomic(dst_p);
kunmap_atomic(src_p);
bio_advance_iter(src, &src_iter, bytes);
bio_advance_iter(dst, &dst_iter, bytes);
}
}
static struct bio *
mt_clone(struct bio *bio) {
struct bio *clone;
clone = bio_clone_bioset(bio, GFP_KERNEL, NULL);
if (!clone) {
return NULL;
}
if (bio_alloc_pages(clone, GFP_KERNEL)) {
bio_put(clone);
return NULL;
}
clone->bi_private = bio;
if (bio_data_dir(bio) == WRITE) {
mt_copy(clone, bio);
}
return clone;
}
static int
mt_map(struct dm_target *ti, struct bio *bio) {
struct mt_private *mdt = (struct mt_private *) ti->private;
bio->bi_bdev = mdt->dev->bdev;
bio = mt_clone(bio);
submit_bio(bio->bi_rw, bio);
return DM_MAPIO_SUBMITTED;
}
This, however, does not work.
When I submit_bio() using the cloned bio I do not get the .end_io call and the calling task becomes blocked ("INFO: task mount:488 blocked for more than 120 seconds."). This with a READ request consisting of a single iovec (1024 bytes). In this case, of course the in buffers do not need copying because they should be overwritten; I need to copy back the incoming data unto the original buffers after the request has completed... but I don't get there.
I'm quite evidently missing some piece, but I'm unable to understand what.
Note: I didn't do any optimization (e.g.: use smarter allocation strategies) specifically because I need to get the basics first.
Note: I corrected a mistake (thanks #RuslanRLaishev), unfortunately ininfluent; see my own answer.
It's correct ?
if (bio_alloc_pages(**bio**, GFP_KERNEL)) {
bio_put(clone);
return NULL;
}
or
if (bio_alloc_pages(**clone**, GFP_KERNEL)) {
bio_put(bio);
return NULL;
}
It turns out bio_clone_bioset() and friends do not copy the callback address to call when request is over.
Trivial solution is to add clone->bi_end_io = bio->bi_end_io; before the end of mt_clone().
Unfortunately this is not enough to make the code functional because it turns out upper layers can spawn thousands of inflight requests (i.e.: requests queued and preprocessed before the previous ones complete) leading to memory starvation. Trying to slow upper layers by returning DM_MAPIO_REQUEUE does not seem to work (see: https://unix.stackexchange.com/q/410525/130498). This has nothing to do with current question, however.
I write a driver in windows, and I need disk drive serial number, for user mode I found this ansver.
My question is it possible to translate the above code to kernel mode, and how? Is WMI query available in filter driver? Sample code can greatly help.
EDIT:
I found here this code, but how I rewrite him for get serial number?
void GetSmbios()
{
NTSTATUS status;
GUID smbiosGUID = SMBIOS_DATA_GUID; // defined in wmiguid.h
PVOID wmiObject = NULL;
PWNODE_ALL_DATA dataBuffer;
ULONG bufferSize;
int TAG_SMBIOS = 'smbi';
//
// Get a WMI block handle to the SMBIOS_DATA_GUID
//
status = IoWMIOpenBlock((GUID *)&smbiosGUID, WMIGUID_QUERY,
&wmiObject);
if (!NT_SUCCESS(status))
{
return status;
}
//
// Determine how much space is required for the data
//
status = IoWMIQueryAllData(wmiObject, &bufferSize, NULL);
if (status != STATUS_BUFFER_TOO_SMALL)
{
ObDereferenceObject(wmiObject);
return status;
}
//
// Allocate the necessary storage. This space must come out of NP-pool
//
dataBuffer = ExAllocatePoolWithTag(
NonPagedPool,
bufferSize,
TAG_SMBIOS);
if (dataBuffer == NULL)
{
ObDereferenceObject(wmiObject);
return STATUS_INSUFFICIENT_RESOURCES;
}
}
After allocating memory, I believe you need to call IoWMIQueryAllData() again, this time passing dataBuffer.
SMBIOS doesn't seem related to disk drives, so you'll want a different GUID to pass to IoWMIOpenBlock(). Perhaps this one ({BF253431-1E4D-4F57-00E7-64B2CACC801E}), since your user-mode example and others query Win32_PhysicalMedia to get SerialNumber.
However, this references a (presumably user-mode) DLL that is the provider for Win32_PhysicalMedia. So this may not be accessible in kernel-mode.
But it also gives a hint how to get the information from kernel-mode: IOCTLs. It mentions IOCTL_SMART_GET_VERSION, which should be just SMART_GET_VERSION, and here's an example:
(in user-mode, but you should be able to do similar from kernel-mode using ZwDeviceIoControlFile()). Note it follows up with another ioctl command, SMART_RCV_DRIVE_DATA, to get the serial number.
Another ioctl that sounds promising (and more general) is IOCTL_STORAGE_QUERY_PROPERTY, with the input STORAGE_PROPERTY_QUERY.PropertyId set to StorageDeviceProperty, so the output will be a STORAGE_DEVICE_DESCRIPTOR structure, which has field SerialNumberOffset:
Specifies the byte offset from the beginning of the structure to a null-terminated ASCII string that contains the device's serial number. If the device has no serial number, this member is zero.
FILE_FS_VOLUME_INFORMATION contains field VolumeSerialNumber. This data structure might be retrieved with ZwQueryVolumeInformationFile(... FileFsVolumeInformation).
That requires a handle to the volume or a file/directory in the volume. If that's not feasible, but you have a DEVICE_OBJECT, you might try building your own IRP with IRP_MJ_QUERY_VOLUME_INFORMATION and sending it with IoCallDriver(), though I don't know if that's sanctioned -- the docs say such a "request is sent by the I/O Manager."
We have a Real Time Operating System which offers Inter-Task-Communication by so called Mailboxes.
A Mailbox is described by a Handle of type RTKMailbox.
The API looks like:
int RTKPut(RTKMailbox h, const void* data);
int RTKGet(RTKMailbox h, void* data);
The size of data is known by the Mailbox. Data transfer could be thought as doing a memcpy from sender to receiver.
Imagine I have a Producer-Task and a Consumer-Task; is it a good idea to send a shared_ptr by that system?
Since the Mailbox does not know a shared_ptr my idea is to wrap the shared_ptr in a transport structure.
The code could look like:
class MyData {
//...
};
struct TransportWrapper {
void BeforePut();
void AfterGet();
std::shared_ptr<MyData> Data;
TransportWrapper() {}
TransportWrapper(std::shared_ptr<MyData>& _data) : Data(_data)
{}
};
void Send(RTKMailbox mbHandle, std::shared_ptr<MyData>& data)
{
TransportWrapper wrap(data);
wrap.BeforePut();
RTKPut(mbHandle, &wrap);
}
std::shared_ptr<MyData> Receive(RTKMailbox mbHandle)
{
TransportWrapper wrap;
RTKGet(mbHandle, &wrap);
wrap.AfterGet();
return wrap.Data;
}
What do I have to do in BeforePut to prevent the shared_ptr to be deleted if the Lifetime of the wrapper ends?
What do I have to do in AfterGet to restore the shared_ptr to the state it had before Put?
Regards Andreas
Your example code won't work, you can't just memcpy a shared_ptr because all that does is copy the pointers it contains, it doesn't make a new copy of the shared_ptr and increase the reference count. You cannot use memcpy with objects that have non-trivial constructors or destructors.
Assuming the sender and receiver share an address space (because otherwise this is pretty much impossible to do via your mailbox API, you need shared memory), you need to increase the shared_ptr's reference count on the sender side, to ensure that the sender doesn't drop its last reference to the owned object and delete it before the receiver has received it. Then the receiver has to decrease the reference count, so they need to coordinate.
If delivery to a mailbox is asynchronous (i.e. the sender does not block until delivery is complete and the receiver has received the data) you can't do that with local variables in the Send function, because those variables will go out of scope as soon as the RTKPut call returns, which will decrease the reference count (and maybe destroy the data) before the receiver has got it.
The simplest way to solve that is to create a new shared_ptr on the heap and transfer its address.
void Send(RTKMailbox mbHandle, const std::shared_ptr<MyData>& data)
{
std::shared_ptr<MyData>* p = new std::shared_ptr<MyData>(data);
if (RTKPut(mbHandle, &p) != success)
{
delete p;
// deal with it
}
}
std::shared_ptr<MyData> Receive(RTKMailbox mbHandle)
{
std::shared_ptr<MyData>* p = nullptr;
if (RTKGet(mbHandle, &p) == success)
{
auto sp = *p;
delete p;
return sp;
}
// else deal with it
}
This assumes that if RTKPut returns successfully then delivery will not fail, otherwise you leak the shared_ptr created on the heap, and will never delete the object it owns.
Can anybody tell me why the following code doesn't work? I don't get any compiler errors.
short value = 10;
SetProp(hCtl, "value", (short*) value);
The third parameter is typed as a HANDLE, so IMO to meet the explicit contract of the function you should save the property as a HANDLE by allocating a HGLOBAL memory block. However, as noted in the comments below, MSDN states that any value can be specified, and indeed when I try it on Windows 7 using...
SetProp(hWnd, _T("TestProp"), (HANDLE)(10)); // or (HANDLE)(short*)(10)
...
(short)GetProp(hWnd, _T("TestProp"));
... I get back 10 from GetProp. I suspect somewhere between your SetProp and GetProp one of two things happens: (1) the value of hWnd is different -- you're checking a different window or (2) a timing issue -- the property hasn't been set yet or had been removed.
If you wanted to use an HGLOBAL instead to follow the specific types of the function signature, you can follow this example in MSDN.
Even though a HANDLE is just a pointer, it's a specific data type that is allocated by calls into the Windows API. Lots of things have handles: icons, cursors, files, ... Unless the documentation explicitly states otherwise, to use a blob of data such as a short when the function calls for a HANDLE, you need a memory handle (an HGLOBAL).
The sample code linked above copies data as a string, but you can instead set it as another data type:
// TODO: Add error handling
hMem = GlobalAlloc(GPTR, sizeof(short));
lpMem = GlobalLock(hMem);
if (lpMem != NULL)
{
*((short*)lpMem) = 10;
GlobalUnlock(hMem);
}
To read it back, when you GetProp to get the HANDLE you must lock it to read the memory:
// TODO: Add error handling
short val;
hMem = (HGLOBAL)GetProp(hwnd, ...);
if (hMem)
{
lpMem = GlobalLock(hMem);
if (lpMem)
{
val = *((short*)lpMem);
}
}
I would create the short on the heap, so that it continues to exist, or perhaps make it global, which is perhaps what you did. Also the cast for the short address needs to be void *, or HANDLE.
I am building a FreeRTOS application. I created a module which registers a freeRTOS queue handle from another module and when an interrupt in this module module occurs, it sends a message to all the registered queues. But it seems I am able to send the message from the queue but not able to receive it at the other module.
Here is my code.
remote module:-
CanRxMsg RxMessage;
can_rx0_queue = xQueueCreate( 10, sizeof(CanRxMsg) ); // can_rx0_queue is globally defined
// Register my queue with can module
if (registerRxQueueWithCAN(can_rx0_queue) == -1)
{
TurnLedRed();
}
while(1)
{
if(can_rx0_queue){
while( xQueueReceive( can_rx0_queue, ( void * ) &RxMessage, portMAX_DELAY))
{
}
.....
Here is the registration module
#define MAX_NUMBER_OF_RX_QUEUES 2
//xQueueHandle rxQueueStore[MAX_NUMBER_OF_RX_QUEUES];
typedef struct QUEUE_REGISTRY_ITEM
{
// signed char *pcQueueName;
xQueueHandle xHandle;
} xQueueRegistryItem;
xQueueRegistryItem rxQueueStore[MAX_NUMBER_OF_RX_QUEUES];
int numberOfQueuesRegistered;
#define cError -1
#define cSuccess 0
void processInterrupt()
{
for(int i=0; i < numberOfQueuesRegistered; i++)
{
if(xQueueSendFromISR(rxQueueStore[i].xHandle,(void *) &RxMessage,&tmp) != pdTRUE)
TurnLedRed();
if(tmp)resched_needed = pdTRUE;
}
portEND_SWITCHING_ISR(resched_needed);
}
int registerRxQueueWithCAN(xQueueHandle myQueue)
{
if(numberOfQueuesRegistered == MAX_NUMBER_OF_RX_QUEUES)
{
// Over Flow of registerations
TurnLedRed();
return cError;
}else
{
rxQueueStore[numberOfQueuesRegistered].xHandle = myQueue;
numberOfQueuesRegistered++;
}
return cSuccess;
}
Few points:-
xQuehandle is typdefed to "void *"
The code works if remove the registration thing and just do with directly pointer of queue in xQueueSendFromISR if I take the pointer by extern.
Any advice or information required?
At first glance I cannot see anything obviously wrong. The problem might be outside of the code you have shown, like how is can_rx0_queue declared, how is the interrupt entered, which port are you using, etc.
There is a FreeRTOS support forum, linked to from the FreeRTOS home page http://www.FreeRTOS.org
Regards.
I think Richard is right. The problem could be issues that are not within your code that you have posted here.
Are you calling any form of suspension on the receiving Task that is waiting on the Queue? When you invoke a vTaskSuspend() on a Task that is blocked waiting on a Queue, the Task that is suspended will be moved to the pxSuspendedTaskList and it will "forget" that it is waiting on an Event Queue because the pvContainer of xEventListItem in that Task will be set to NULL.
You might want to check if your receiving Task is ever suspended while waiting on a Queue. Hope that helped. Cheers!
Your shared memory should at least be declared volatile:
volatile xQueueRegistryItem rxQueueStore[MAX_NUMBER_OF_RX_QUEUES] ;
volatile int numberOfQueuesRegistered ;
otherwise the compiler may optimise out read or writes to these because it has no concept of different threads of execution (between the ISR and the main thread).
Also I recall that some PIC C runtime start-up options do not apply zero-initialisation of static data in order to minimise start-up time, if you are using such a start-up, you should explicitly initialise numberOfQueuesRegistered. I would suggest that to do so would be a good idea in any case.
It is not clear from your code that RxMessage in the ISR is not the same as RxMessage in the 'remote module'; they should not be shared, since that would allow the ISR to potentially modify the data while the receiving thread was processing it. If they could be shared, there would ne no reason to have a queue in the first place, since shared memory and a semaphore would suffice.
As a side-note, there is never any need to cast a pointer to void*, and you should generally avoid doing so, since it will prevent the compiler from issuing an error if you were to pass something other than a pointer. The whole point of a void* is rather that it can accept any pointer type.