Related
Hello i created Network driver that uses the uart port to send and recive.
My driver works with some issues. I was able to ping but always afther a few pings i get
ping: sendmsg: No buffer space available driver
I checked the kernel logs but i could not see anything.
this is how i recive data:
struct stm32_port *stm32_port = netdev_priv(my_net);
struct sk_buff *skb;
unsigned char *dma_start;
dma_start = stm32_port->rx_buf + (RX_BUF_L - stm32_port->last_res);
print_hex_dump(KERN_DEBUG, "data: ", DUMP_PREFIX_OFFSET, 16, 1, dma_start, 16, true);
skb = dev_alloc_skb(dma_size + 2);
if (!skb) {
if (printk_ratelimit( ))
printk(KERN_NOTICE "snull rx: low on mem - packet dropped\n");
my_net->stats.rx_dropped++;
//goto error;
}
memcpy(skb_put(skb, dma_size), dma_start, dma_size);
/* Write metadata, and then pass to the receive level */
skb->dev = my_net;
skb->protocol = eth_type_trans(skb, my_net);
skb->ip_summed = CHECKSUM_NONE; // let the OS check the checksum
my_net->stats.rx_packets++;
my_net->stats.rx_bytes += dma_size;
netif_rx(skb);
port->icount.rx += dma_size;
stm32_port->last_res -= dma_size;
if (stm32_port->last_res == 0)
stm32_port->last_res = RX_BUF_L; //dma_count
Here is how i send my data:
struct stm32_port *lp = netdev_priv(ndev);
struct uart_port *port = &lp->port;
struct sk_buff *sk_buff;
struct dma_async_tx_descriptor *desc = NULL;
struct stm32_usart_offsets *ofs = &lp->info->ofs;
unsigned pktlen = skb->len;
dma_cookie_t cookie;
int ret = 0;
//print_hex_dump(KERN_DEBUG, "data: ", DUMP_PREFIX_OFFSET, 16, 1, skb->data, 16, true);
netif_stop_queue(ndev);
sk_buff = skb_get(skb);
if (ofs->icr == UNDEF_REG){
stm32_usart_clr_bits(port, ofs->isr, USART_SR_TC);
}else{
writel_relaxed(USART_ICR_TCCF, port->membase + ofs->icr);
}
memcpy(&lp->tx_buf[0], sk_buff->data, pktlen);
desc = dmaengine_prep_slave_single(lp->tx_ch,
lp->tx_dma_buf,
pktlen,
DMA_MEM_TO_DEV,
DMA_PREP_INTERRUPT);
if (!desc){
goto fallback_err;
}
cookie = dmaengine_submit(desc);
ret = dma_submit_error(cookie);
if (ret) {
/* dma no yet started, safe to free resources */
dmaengine_terminate_async(lp->tx_ch);
goto fallback_err;
}
/* Issue pending DMA TX requests */
dma_async_issue_pending(lp->tx_ch);
stm32_usart_set_bits(port, ofs->cr3, USART_CR3_DMAT);
/* rely on TXE irq (mask or unmask) for sending remaining data */
stm32_usart_tx_interrupt_disable(port);
ndev->stats.tx_packets++;
ndev->stats.tx_bytes += pktlen;
fallback_err:
skb_tx_timestamp(skb);
dev_kfree_skb (skb);
netif_start_queue(ndev);
return NETDEV_TX_OK;
Thanks to #stark i found a solution i free the buffer now with the __kfree_skb() methode because like that the refcount will not be checked.
I've been working on modifying the intel ixgbe kernel driver to function with my PCIe device (FPGA but that's not super important). The kernel and the PCIe device all negotiate quite well, configuration headers are passed along and communication seems to function. However attempting to write DMA_FROM_DEVICE I have a slight problem that I don't understand and I'm hoping for help.
rx_ring->desc = dma_alloc_coherent(dev, ///This function allocates dma space of size size for handle dma on device dev with flag GFP KERNEL
rx_ring->size,
&rx_ring->dma, ///This dma handle may be cast to unsigned integer of the same bus width and given to dev as the DMA base address
GFP_KERNEL);
page = dev_alloc_pages(0);
dma = dma_map_page(rx_ring->dev, page, 0, acc_rx_pg_size(rx_ring), DMA_FROM_DEVICE);
//Writing to the PCI device the base address to place data into.
writel(q_vector->adapter->rx_ring[0]->dma >> 32, q_vector->adapter->hw_region2.hw_addr+0x08+ACC_PCI_IPCONT_DATA_OFFSET);
writel(q_vector->adapter->rx_ring[0]->dma & 0xFFFFFFFF, q_vector->adapter->hw_region2.hw_addr+0x0C+ACC_PCI_IPCONT_DATA_OFFSET);
//This will perfectly read data I place onto the PCIe bus.
rx_ring->desc->wb.upper.length
//This seems to read some garbage memory.
dma_sync_single_range_for_cpu(rx_ring->dev,
rx_buffer->dma,
rx_buffer->page_offset,
acc_rx_bufsz(rx_ring),
DMA_FROM_DEVICE);
unsigned char *va = page_address(page) + rx_buffer->page_offset;
memcpy(__skb_put(skb, size), va, ALIGN(size, sizeof(long)));
//Some code later
dma_sync_single_range_for_device(rx_ring->dev, new_buff->dma,
new_buff->page_offset,
acc_rx_bufsz(rx_ring),
DMA_FROM_DEVICE);
I've tried to purge code down to just the points of interest but here's the brief run down. I allocate space for the dma creating the virtual and bus address via the dma_alloc_coherent function. I create a page of memory for the dma and map this page to the dma via the dev_alloc_pages and dma_map_page commands. I pass the dma bus address to my PCIe device so it can write to the proper offset via the writel commands (I know iowrite32 but this is on redhat).
From here there are 2 ways that the origonal ixgbe driver reads data from the PCIe bus. First it directly reads from the dma's allocated virtual address (desc), but this is only used for configuration information (in the driver I am working off of). The second method is via use page_address(page) to I believe get a virtual address for the page of memory. The problem is there is only garbage memory there.
So here is my confusion. Where is page pointing to and how do I place data into page via the PCI bus? I assumed that dma_map_page would sort of merge the 2 virtual addresses into 1 so my write into the dma's bus address would collide into the page but this doesn't seem to be the case. What base address should my PCI device be writing from to align into this page of memory?
I'm working on redhat, specifically Centos kernel version 3.10.0 which makes for some problems since redhat kernel is very different from base kernel but hopefully someone can help. Thank you for any pointers.
EDIT: Added dma_sync calls which I forgot to include in original post.
EDIT2: Added a more complete code base. As a note I'm still not including some of the struct definitions or top function calls (like probe for instance), but hopefully this will be a lot more complete. Sorry for how long it is.
//These functions are called during configuration
int acc_setup_rx_resources(struct acc_ring *rx_ring)
{
struct device *dev = rx_ring->dev;
int orig_node = dev_to_node(dev);
int numa_node = -1;
int size;
size = sizeof(struct acc_rx_buffer) * rx_ring->count;
if (rx_ring->q_vector)
numa_node = rx_ring->q_vector->numa_node;
rx_ring->rx_buffer_info = vzalloc_node(size, numa_node);
if (!rx_ring->rx_buffer_info)
rx_ring->rx_buffer_info = vzalloc(size);
if (!rx_ring->rx_buffer_info)
goto err;
/* Round up to nearest 4K */
rx_ring->size = rx_ring->count * sizeof(union acc_adv_rx_desc);
rx_ring->size = ALIGN(rx_ring->size, 4096);
set_dev_node(dev, numa_node);
rx_ring->desc = dma_alloc_coherent(dev,
rx_ring->size,
&rx_ring->dma,
GFP_KERNEL);
set_dev_node(dev, orig_node);
if (!rx_ring->desc)
rx_ring->desc = dma_alloc_coherent(dev, rx_ring->size,
&rx_ring->dma, GFP_KERNEL);
if (!rx_ring->desc)
goto err;
rx_ring->next_to_clean = 0;
rx_ring->next_to_use = 0;
return 0;
err:
vfree(rx_ring->rx_buffer_info);
rx_ring->rx_buffer_info = NULL;
dev_err(dev, "Unable to allocate memory for the Rx descriptor ring\n");
return -ENOMEM;
}
static bool acc_alloc_mapped_page(struct acc_ring *rx_ring,
struct acc_rx_buffer *bi)
{
struct page *page = bi->page;
dma_addr_t dma = bi->dma;
if (likely(page))
return true;
page = dev_alloc_pages(0);
if(unlikely(!page)){
rx_ring->rx_stats.alloc_rx_page_failed++;
return false;
}
/* map page for use */
dma = dma_map_page(rx_ring->dev, page, 0,
acc_rx_pg_size(rx_ring), DMA_FROM_DEVICE);
if (dma_mapping_error(rx_ring->dev, dma)) {
__free_pages(page, acc_rx_pg_order(rx_ring));
bi->page = NULL;
rx_ring->rx_stats.alloc_rx_page_failed++;
return false;
}
bi->dma = dma;
bi->page = page;
bi->page_offset = 0;
page_ref_add(page, USHRT_MAX - 1); //This seems to exist in redhat kernel but not 3.10 base kernel... keep?
return true;
}
void acc_alloc_rx_buffers(struct acc_ring *rx_ring, u16 cleaned_count)
{
union acc_adv_rx_desc *rx_desc;
struct acc_rx_buffer *bi;
u16 i = rx_ring->next_to_use;
printk(KERN_INFO "acc Attempting to allocate rx buffers\n");
/* nothing to do */
if (!cleaned_count)
return;
rx_desc = ACC_RX_DESC(rx_ring, i);
bi = &rx_ring->rx_buffer_info[i];
i -= rx_ring->count;
do {
if (!acc_alloc_mapped_page(rx_ring, bi)){
printk(KERN_INFO "acc Failed to allocate and map the page to dma\n");
break;
}
printk(KERN_INFO "acc happily allocated and mapped page to dma\n");
/*
* Refresh the desc even if buffer_addrs didn't change
* because each write-back erases this info.
*/
rx_desc->read.pkt_addr = cpu_to_le64(bi->dma + bi->page_offset);
rx_desc++;
bi++; ///Move to the next buffer
i++;
if (unlikely(!i)) {
rx_desc = ACC_RX_DESC(rx_ring, 0);
bi = rx_ring->rx_buffer_info;
i -= rx_ring->count;
}
/* clear the hdr_addr for the next_to_use descriptor */
rx_desc->read.hdr_addr = 0;
cleaned_count--;
} while (cleaned_count);
i += rx_ring->count;
if (rx_ring->next_to_use != i)
acc_release_rx_desc(rx_ring, i);
}
//This function is called via a napi_schedule command which fires when an MSI interrupt is thrown from my PCIe device (all works fine).
int acc_poll(struct napi_struct *napi, int budget)
{
struct acc_q_vector *q_vector =
container_of(napi, struct acc_q_vector, napi);
struct acc_adapter *adapter = q_vector->adapter;
struct acc_ring *ring;
int per_ring_budget;
bool clean_complete = true;
e_dev_info("Landed in acc_poll\n");
e_dev_info("Attempting to read register space 0x00=%x\t0x04=%x\n", \
readl(q_vector->adapter->hw.hw_addr), readl(q_vector->adapter->hw.hw_addr+0x04));
e_dev_info("Attempting to write to pci ctl\n");
e_dev_info("Target address %.8x%.8x\n",q_vector->adapter->rx_ring[0]->dma >> 32, q_vector->adapter->rx_ring[0]->dma & 0xFFFFFFFF);
e_dev_info("Attempted page address %.8x%.8x\n",virt_to_bus(page_address(q_vector->adapter->rx_ring[0]->rx_buffer_info[0].page)) >> 32, virt_to_bus(page_address(q_vector->adapter->rx_ring[0]->rx_buffer_info[0].page)) & 0xFFFFFFFF);
writeq(0x0000000000000001, q_vector->adapter->hw_region2.hw_addr+ACC_PCI_IPCONT_DATA_OFFSET); //These are supposed to be iowrite64 but it seems iowrite64 is different in redhat and only supports the copy function (to,from,size). yay redhat think different.
writel(q_vector->adapter->rx_ring[0]->dma >> 32, q_vector->adapter->hw_region2.hw_addr+0x08+ACC_PCI_IPCONT_DATA_OFFSET);
writel(q_vector->adapter->rx_ring[0]->dma & 0xFFFFFFFF, q_vector->adapter->hw_region2.hw_addr+0x0C+ACC_PCI_IPCONT_DATA_OFFSET);
writel(virt_to_bus(page_address(q_vector->adapter->rx_ring[0]->rx_buffer_info[0].page)) >> 32, q_vector->adapter->hw_region2.hw_addr+0x10+ACC_PCI_IPCONT_DATA_OFFSET);
writel(virt_to_bus(page_address(q_vector->adapter->rx_ring[0]->rx_buffer_info[0].page)) & 0xFFFFFFFF, q_vector->adapter->hw_region2.hw_addr+0x14+ACC_PCI_IPCONT_DATA_OFFSET);
writeq(0xFF00000000000000, q_vector->adapter->hw_region2.hw_addr+0x18+ACC_PCI_IPCONT_DATA_OFFSET);
writeq(0x0000000CC0000000, q_vector->adapter->hw_region2.hw_addr+0x20+ACC_PCI_IPCONT_DATA_OFFSET);
writeq(0x0000000CC0000000, q_vector->adapter->hw_region2.hw_addr+0x28+ACC_PCI_IPCONT_DATA_OFFSET);
writeq(0x0003344000005500, q_vector->adapter->hw_region2.hw_addr+0x30+ACC_PCI_IPCONT_DATA_OFFSET);
//Send the start command to the block
writeq(0x0000000000000001, q_vector->adapter->hw_region2.hw_addr);
acc_for_each_ring(ring, q_vector->tx)
clean_complete &= !!acc_clean_tx_irq(q_vector, ring);
if (q_vector->rx.count > 1)
per_ring_budget = max(budget/q_vector->rx.count, 1);
else
per_ring_budget = budget;
acc_for_each_ring(ring, q_vector->rx){
e_dev_info("Calling clean_rx_irq\n");
clean_complete &= acc_clean_rx_irq(q_vector, ring,
per_ring_budget);
}
/* If all work not completed, return budget and keep polling */
if (!clean_complete)
return budget;
e_dev_info("Clean complete\n");
/* all work done, exit the polling mode */
napi_complete(napi);
if (adapter->rx_itr_setting & 1)
acc_set_itr(q_vector);
if (!test_bit(__ACC_DOWN, &adapter->state))
acc_irq_enable_queues(adapter, ((u64)1 << q_vector->v_idx));
e_dev_info("Exiting acc_poll\n");
return 0;
}
static bool acc_clean_rx_irq(struct acc_q_vector *q_vector,
struct acc_ring *rx_ring,
const int budget)
{
printk(KERN_INFO "acc Entered clean_rx_irq\n");
unsigned int total_rx_bytes = 0, total_rx_packets = 0;
u16 cleaned_count = acc_desc_unused(rx_ring); /// First pass this is count-1 because ntc and ntu are 0 so this is 512-1=511
printk(KERN_INFO "acc RX irq Clean count = %d\n", cleaned_count);
do {
union acc_adv_rx_desc *rx_desc;
struct sk_buff *skb;
/* return some buffers to hardware, one at a time is too slow */
if (cleaned_count >= ACC_RX_BUFFER_WRITE) { //When the clean count is >16 allocate some more buffers to get the clean count down. First pass this happens.
acc_alloc_rx_buffers(rx_ring, cleaned_count);
cleaned_count = 0;
}
rx_desc = ACC_RX_DESC(rx_ring, rx_ring->next_to_clean);
printk(KERN_INFO "acc inside RX do while, acquired description\n");
printk(KERN_INFO "acc Everything I can about the rx_ring desc (acc_rx_buffer). status_error=%d\t \
length=%d\n", rx_desc->wb.upper.status_error, rx_desc->wb.upper.length);
if (!acc_test_staterr(rx_desc, ACC_RXD_STAT_DD))
break;
printk(KERN_INFO "acc inside RX past status_error check\n");
/*
* This memory barrier is needed to keep us from reading
* any other fields out of the rx_desc until we know the
* RXD_STAT_DD bit is set
*/
rmb();
/* retrieve a buffer from the ring */
skb = acc_fetch_rx_buffer(rx_ring, rx_desc);
/* exit if we failed to retrieve a buffer */
if (!skb)
break;
printk(KERN_INFO "acc successfully retrieved a buffer\n");
cleaned_count++;
/* place incomplete frames back on ring for completion */
if (acc_is_non_eop(rx_ring, rx_desc, skb))
continue;
/* verify the packet layout is correct */
if (acc_cleanup_headers(rx_ring, rx_desc, skb))
continue;
/* probably a little skewed due to removing CRC */
total_rx_bytes += skb->len;
/* populate checksum, timestamp, VLAN, and protocol */
acc_process_skb_fields(rx_ring, rx_desc, skb);
acc_rx_skb(q_vector, skb); ///I believe this sends data to the kernel network stuff and then the generic OS
/* update budget accounting */
total_rx_packets++;
} while (likely(total_rx_packets < budget));
printk(KERN_INFO "acc rx irq exited the while loop\n");
u64_stats_update_begin(&rx_ring->syncp);
rx_ring->stats.packets += total_rx_packets;
rx_ring->stats.bytes += total_rx_bytes;
u64_stats_update_end(&rx_ring->syncp);
q_vector->rx.total_packets += total_rx_packets;
q_vector->rx.total_bytes += total_rx_bytes;
if (cleaned_count)
acc_alloc_rx_buffers(rx_ring, cleaned_count);
printk(KERN_INFO "acc rx irq returning happily\n");
return (total_rx_packets < budget);
}
static struct sk_buff *acc_fetch_rx_buffer(struct acc_ring *rx_ring,
union acc_adv_rx_desc *rx_desc)
{
struct acc_rx_buffer *rx_buffer;
struct sk_buff *skb;
struct page *page;
printk(KERN_INFO "acc Attempting to fetch rx buffer\n");
rx_buffer = &rx_ring->rx_buffer_info[rx_ring->next_to_clean];
page = rx_buffer->page; //This page is set by I think acc_add_rx_frag... hard to tell. yes the page is created there and kind of linked to the dma via dma_map_page
prefetchw(page); ///Prefetch the page cacheline for writing
skb = rx_buffer->skb; ///This does the mapping between skb and dma page table I believe.
if (likely(!skb)) {
printk(KERN_INFO "acc attempting to allocate netdrv space for page.\n");
void *page_addr = page_address(page) + //get the virtual page address of this page.
rx_buffer->page_offset;
/* prefetch first cache line of first page */
prefetch(page_addr);
#if L1_CACHE_BYTES < 128
prefetch(page_addr + L1_CACHE_BYTES);
#endif
/* allocate a skb to store the frags */
skb = netdev_alloc_skb_ip_align(rx_ring->netdev,
ACC_RX_HDR_SIZE);
if (unlikely(!skb)) {
rx_ring->rx_stats.alloc_rx_buff_failed++;
return NULL;
}
/*
* we will be copying header into skb->data in
* pskb_may_pull so it is in our interest to prefetch
* it now to avoid a possible cache miss
*/
prefetchw(skb->data);
/*
* Delay unmapping of the first packet. It carries the
* header information, HW may still access the header
* after the writeback. Only unmap it when EOP is
* reached
*/
if (likely((rx_desc, ACC_RXD_STAT_EOP)))
goto dma_sync;
ACC_CB(skb)->dma = rx_buffer->dma;
} else {
if (acc_test_staterr(rx_desc, ACC_RXD_STAT_EOP))
acc_dma_sync_frag(rx_ring, skb);
dma_sync:
/* we are reusing so sync this buffer for CPU use */
printk(KERN_INFO "acc attempting to sync the dma and the device.\n");
dma_sync_single_range_for_cpu(rx_ring->dev, //Sync to the pci device, this dma buffer, at this page offset, this ring, for device to DMA transfer
rx_buffer->dma,
rx_buffer->page_offset,
acc_rx_bufsz(rx_ring),
DMA_FROM_DEVICE);
}
/* pull page into skb */
if (acc_add_rx_frag(rx_ring, rx_buffer, rx_desc, skb)) {
//This is again temporary to try and create blockers around the problem.
return skb;
/* hand second half of page back to the ring */
acc_reuse_rx_page(rx_ring, rx_buffer);
} else if (ACC_CB(skb)->dma == rx_buffer->dma) {
/* the page has been released from the ring */
ACC_CB(skb)->page_released = true;
} else {
/* we are not reusing the buffer so unmap it */
dma_unmap_page(rx_ring->dev, rx_buffer->dma,
acc_rx_pg_size(rx_ring),
DMA_FROM_DEVICE);
}
/* clear contents of buffer_info */
rx_buffer->skb = NULL;
rx_buffer->dma = 0;
rx_buffer->page = NULL;
printk(KERN_INFO "acc returning from fetch_rx_buffer.\n");
return skb;
}
static bool acc_add_rx_frag(struct acc_ring *rx_ring,
struct acc_rx_buffer *rx_buffer,
union acc_adv_rx_desc *rx_desc,
struct sk_buff *skb)
{
printk(KERN_INFO "acc Attempting to add rx_frag from page.\n");
struct page *page = rx_buffer->page;
unsigned int size = le16_to_cpu(rx_desc->wb.upper.length);
#if (PAGE_SIZE < 8192)
unsigned int truesize = acc_rx_bufsz(rx_ring);
#else
unsigned int truesize = ALIGN(size, L1_CACHE_BYTES);
unsigned int last_offset = acc_rx_pg_size(rx_ring) -
acc_rx_bufsz(rx_ring);
#endif
if ((size <= ACC_RX_HDR_SIZE) && !skb_is_nonlinear(skb)) {
printk(KERN_INFO "acc Inside the size check.\n");
unsigned char *va = page_address(page) + rx_buffer->page_offset;
printk(KERN_INFO "page:%p\tpage_address:%p\tpage_offset:%d\n",page,page_address(page),rx_buffer->page_offset);
printk(KERN_INFO "acc First 4 bytes of string:%x %x %x %x\n",va[0],va[1],va[2],va[3]); //FIXME: I can now read this page table but there is still no meaningful data in it. (appear to be reading garbage)
printk(KERN_INFO "acc 32 bytes in:%x %x %x %x\n",va[32],va[33],va[34],va[35]);
return true;
memcpy(__skb_put(skb, size), va, ALIGN(size, sizeof(long)));
/* we can reuse buffer as-is, just make sure it is local */
if (likely(page_to_nid(page) == numa_node_id()))
return true;
/* this page cannot be reused so discard it */
put_page(page);
return false;
}
skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page,
rx_buffer->page_offset, size, truesize);
/* avoid re-using remote pages */
if (unlikely(page_to_nid(page) != numa_node_id()))
return false;
#if (PAGE_SIZE < 8192)
/* if we are only owner of page we can reuse it */
if (unlikely(page_count(page) != 1))
return false;
/* flip page offset to other buffer */
rx_buffer->page_offset ^= truesize;
/*
* since we are the only owner of the page and we need to
* increment it, just set the value to 2 in order to avoid
* an unecessary locked operation
*/
atomic_set(&page->_count, 2);
#else
/* move offset up to the next cache line */
rx_buffer->page_offset += truesize;
if (rx_buffer->page_offset > last_offset)
return false;
/* bump ref count on page before it is given to the stack */
get_page(page);
#endif
return true;
}
I've got problems to read and store the received data by a TCP server. I'm using the LWIP library and the NUCLEO-F746ZG board. I suppose that I have to get the data when I do es->p. I've read that, you have to use the payload but I don't know how to implement it well in mi receive callback:
static err_t tcp_echoserver_recv(void *arg, struct tcp_pcb *tpcb, struct pbuf *p, err_t err)
{
struct tcp_echoserver_struct *es;
err_t ret_err;
u32_t *tempPtr;
LWIP_ASSERT("arg != NULL",arg != NULL);
es = (struct tcp_echoserver_struct *)arg;
/* if we receive an empty tcp frame from client => close connection */
if (p == NULL)
{
/* remote host closed connection */
es->state = ES_CLOSING;
if(es->p == NULL)
{
/* we're done sending, close connection */
tcp_echoserver_connection_close(tpcb, es);
}
else
{
/* we're not done yet */
/* acknowledge received packet */
tcp_sent(tpcb, tcp_echoserver_sent);
/* send remaining data*/
tcp_echoserver_send(tpcb, es);
}
ret_err = ERR_OK;
}
/* else : a non empty frame was received from client but for some reason err != ERR_OK */
else if(err != ERR_OK)
{
/* free received pbuf*/
if (p != NULL)
{
es->p = NULL;
pbuf_free(p);
}
ret_err = err;
}
else if(es->state == ES_ACCEPTED)
{
/* first data chunk in p->payload */
es->state = ES_RECEIVED;
/* store reference to incoming pbuf (chain) */
es ->p = p; //I canĀ“t convert this into a String
/* initialize LwIP tcp_sent callback function */
tcp_sent(tpcb, tcp_echoserver_sent);
/* send back the received data (echo) */
tcp_echoserver_send(tpcb, es);
ret_err = ERR_OK;
}
else if (es->state == ES_RECEIVED)
{
/* more data received from client and previous data has been already sent*/
if(es->p == NULL)
{
//es->p = p;
/* send back received data */
tcp_echoserver_send(tpcb, es);
}
else
{
struct pbuf *ptr;
/* chain pbufs to the end of what we recv'ed previously */
//ptr = es->p;
//pbuf_chain(ptr,p);
}
ret_err = ERR_OK;
}
else if(es->state == ES_CLOSING)
{
/* odd case, remote side closing twice, trash data */
//tcp_recved(tpcb, p->tot_len);
//es->p = NULL;
//pbuf_free(p);
ret_err = ERR_OK;
}
else
{
/* unkown es->state, trash data */
//tcp_recved(tpcb, p->tot_len);
//es->p = NULL;
//pbuf_free(p);
ret_err = ERR_OK;
}
return ret_err;
}
If anyone knows how to extract the data as a string it would help me a lot.
thank you in advance!!
You need to memcpy the pyload to your char array. That is the only way of getting the payload
char str[somelegth];
memcpy(str, p -> payload, p -> len);
Program attached:
Those codes are written by me based on the TI uart.c driver library.
while(1){
//check if message on Queue -> read or check UART input
if(uxQueueMessagesWaiting( UART_TASKQ ) != 0){ //may have bugs
// deQueue
xQueueReceive( UART_TASKQ, &UARTTaskHandle, 0x0A );
//do the task's mission using the data in the stucture(put by control
task)
//Print out the input data.
//**********debugging data
/*
testPointer = UARTTaskHandle->dataBuffer;
testAmount = UARTTaskHandle->dataSize;
while(testAmount){
if(*testPointer != 1){
error = error + 1;
}
if(*(testPointer + 1) != 2){
error = error + 1;
}
if(*(testPointer + 2) != 3){
error = error + 1;
}
if(*(testPointer + 3) != 4){
error = error + 1;
}
if(*(testPointer + 4) != 5){
error = error + 1;
}
if(*(testPointer + 5) != 6){
error = error + 1;
}
if(*(testPointer + 6) != 7){
error = error + 1;
}
if(*(testPointer + 7) != 8){
error = error + 1;
}
testPointer = testPointer + 8;
testAmount = testAmount - 8;
}
*/
if(UART_write(UART_Handle, UARTTaskHandle->dataBuffer, UARTTaskHandle->dataSize, 0xff ) >= 0){
UARTwriteCount = UARTwriteCount + 1;
}
//let control task take new command
//free allocated memory
free(UARTTaskHandle->dataBuffer);
free(UARTTaskHandle); // free memory space
//(above is code using UART)
//here are UART driver code:
unsigned long UARTStatus(unsigned long ulBase){
ASSERT(UARTBaseValid(ulBase));
return(HWREG(ulBase + UART_O_FR));
}
//*****************************************
//UART_ISR
//Interrupt service routine for
//the UART read and write process
//*****************************************
void UART_ISR(){
//read FIFO full or read time out
if(UARTIntStatus(UART_Handle->UART_PORT,false) & (UART_INT_RX | UART_INT_RT)){
UARTIntClear(UART_Handle->UART_PORT, UART_INT_RX | UART_INT_RT); //clear INT flag
while (!(UARTStatus(UART_Handle->UART_PORT) & UART_FR_RXFE)){
//data reading
*UART_Handle->pCurrentRead = UARTCharGet(UART_Handle->UART_PORT); //read autoly clear INT
UART_Handle->pCurrentRead++;
UART_Handle->ReadLength--;
//adjust code here:
if(UART_Handle->ReadLength == 0){
break;
}
}
//check if read certain bytes finished
if(UART_Handle->ReadLength == 0){
memcpy(UART_Handle->dataput, UART_Handle->pReadBuf,UART_Handle->ReadLengthcpy); // copy data back
xSemaphoreGiveFromISR( UART_Handle->UARTRead_Semaphore, &xHigherPriorityTaskWoken );// release semaphore
portYIELD_FROM_ISR( xHigherPriorityTaskWoken );//forcing context exchange
}
}
//send FIFO empty
if(UARTIntStatus(UART_Handle->UART_PORT,false) & UART_INT_TX){
UARTIntClear(UART_Handle->UART_PORT, UART_INT_TX); //clear INT flag
if(UART_Handle->WriteLength == BUFFEMPTY){
UART_Handle->UART_SendComplete = true;
xSemaphoreGiveFromISR( UART_Handle->UARTWrite_Semaphore, &xHigherPriorityTaskWoken );// release semaphore
portYIELD_FROM_ISR( xHigherPriorityTaskWoken );//forcing context exchange
}
//putting data into send FIFO
if(UART_Handle->WriteLength > FIFOMAX){
for( Cindex = 0 ; Cindex < FIFOMAX ;){
if(UARTCharPutNonBlocking(UART_Handle->UART_PORT, *(UART_Handle->pCurrentWrite))){//write autoly clear INT
(UART_Handle->pCurrentWrite) = (UART_Handle->pCurrentWrite) + 1;
(UART_Handle->WriteLength) = (UART_Handle->WriteLength) - 1;
Cindex = Cindex + 1;
UART_Handle->sentCount = UART_Handle->sentCount + 1;
}
}
}else{
templength = UART_Handle->WriteLength;
for( Cindex = 0; Cindex < templength ;){
if(UARTCharPutNonBlocking(UART_Handle->UART_PORT, *(UART_Handle->pCurrentWrite))){//write autoly clear INT
(UART_Handle->pCurrentWrite) = (UART_Handle->pCurrentWrite) + 1;
(UART_Handle->WriteLength) = (UART_Handle->WriteLength) - 1;
Cindex = Cindex + 1;
UART_Handle->sentCount = UART_Handle->sentCount + 1;
}
}
}
}
}
//*****************************************
//UART_write
//write certain length of data to UART port
//*****************************************
int32_t UART_write( UART_STATE *UART_Handle, uint8_t *pData, uint32_t length, uint32_t time_out ){
while(!UART_Handle->UART_SendComplete); //debugging purpose
UART_Handle->UART_SendComplete = false;//debugging purpose
UART_Handle->WriteLength = length;
if(UART_Handle->WriteLength <= UART_Handle->WriteBufSize){
UARTIntClear(UART_Handle->UART_PORT, UART_INT_TX); //clear INT flag
memcpy(UART_Handle->pWriteBuf,pData,UART_Handle->WriteLength); //copy data into writebuff
UART_Handle->pCurrentWrite = UART_Handle->pWriteBuf;
//putting data into send FIFO
if(UART_Handle->WriteLength > FIFOMAX){
// if
for( Cindex = 0 ; Cindex < FIFOMAX ;){
if(UARTCharPutNonBlocking(UART_Handle->UART_PORT, *(UART_Handle->pCurrentWrite))){//write autoly clear INT
(UART_Handle->pCurrentWrite) = (UART_Handle->pCurrentWrite) + 1;
(UART_Handle->WriteLength) = (UART_Handle->WriteLength) - 1;
Cindex = Cindex + 1;
UART_Handle->sentCount = UART_Handle->sentCount + 1;
}
}
}else{
for( Cindex = 0 ; Cindex < FIFOMAX ;){
if(UARTCharPutNonBlocking(UART_Handle->UART_PORT, *(UART_Handle->pCurrentWrite))){//write autoly clear INT
(UART_Handle->pCurrentWrite) = (UART_Handle->pCurrentWrite) + 1;
(UART_Handle->WriteLength) = (UART_Handle->WriteLength) - 1;
Cindex = Cindex + 1;
UART_Handle->sentCount = UART_Handle->sentCount + 1;
}
}
}
//start sending
UARTEnable(UART_Handle->UART_PORT);
if(UART_Handle->UARTWrite_Semaphore != NULL ) {
if(xSemaphoreTake(UART_Handle->UARTWrite_Semaphore, time_out/(portTICK_PERIOD_MS) ) == pdTRUE){
reValue = WRITESUCCESS; //wait return till write complete
}else{
reValue = WRITETIMEOUT; // timeout (ms)
}
}else{
while(1); //no Semaphore
}
return reValue;
}else{
return FAILURE; //wrong length
}
}
//*****************************************
//UART_read
//read certain length of data from UART port
//*****************************************
int32_t UART_read(UART_STATE *UART_Handle, uint8_t *pData, uint32_t length, uint32_t time_out){
//later added part
UARTDisable(UART_Handle->UART_PORT); //clearUART
UARTFIFOEnable(UART_Handle->UART_PORT);
//
UART_Handle->ReadLength = length; // set readlength
UART_Handle->ReadLengthcpy = length;
if(UART_Handle->ReadLength <= UART_Handle->ReadBufSize){
UARTIntClear(UART_Handle->UART_PORT, UART_INT_RX | UART_INT_RT); //clear INT flag
UART_Handle->dataput = pData; //store the destination buffer address
UART_Handle->pCurrentRead = UART_Handle->pReadBuf; //set current read
UARTEnable(UART_Handle->UART_PORT); //start receiving
//suspend before read ISR finish whole process
if(UART_Handle->UARTRead_Semaphore != NULL ) {
if(xSemaphoreTake(UART_Handle->UARTRead_Semaphore, time_out/(portTICK_PERIOD_MS) ) == pdTRUE){
reValue = READSUCCESS; //wait return till write complete
}else{
reValue = READTIMEOUT; // timeout (ms)
}
}else{
while(1); //no Semaphore
}
return reValue;
}else{
return FAILURE; //wrong length
}
}
//*****************************************
//UART_open
//open UART for certain port and bandrate
//*****************************************
UART_HANDLE UART_open(uint32_t UART_port, uint32_t UART_portperiph, uint32_t UART_baudrate){
//initialize structure
UART_Handle = (UART_HANDLE)malloc(sizeof(UART_STATE));
UART_Handle->ReadBufSize = UARTBUFFERSIZE;
UART_Handle->WriteBufSize = UARTBUFFERSIZE;
UART_Handle->UART_PORT = UART_port;
UART_Handle->UART_PORTPERIPH = UART_portperiph;
UART_Handle->UART_BRATE = UART_baudrate;
UART_Handle->pWriteBuf = (uint8_t*)malloc(UART_Handle->WriteBufSize * sizeof(uint8_t));
UART_Handle->pReadBuf = (uint8_t*)malloc(UART_Handle->ReadBufSize * sizeof(uint8_t));
UART_Handle->pCurrentWrite = UART_Handle->pWriteBuf;
UART_Handle->pCurrentRead = UART_Handle->pReadBuf;
UART_Handle->UARTWrite_Semaphore = NULL;
UART_Handle->UARTRead_Semaphore = NULL;
UART_Handle->UARTprotect_Semaphore = NULL;
UART_Handle->UART_SendComplete = true;
UART_Handle->sentCount = 0;//debugging purpose
vSemaphoreCreateBinary( UART_Handle->UARTWrite_Semaphore ); //semaphore create
vSemaphoreCreateBinary( UART_Handle->UARTRead_Semaphore ); //semaphore create
// vSemaphoreCreateBinary( UART_Handle->UARTprotect_Semaphore ); //debugging purpose
xSemaphoreTake( UART_Handle->UARTRead_Semaphore, portMAX_DELAY ); //semaphore take
xSemaphoreTake( UART_Handle->UARTWrite_Semaphore, portMAX_DELAY ); //semaphore take
// Enable Peripheral Clocks
MAP_PRCMPeripheralClkEnable(UART_Handle->UART_PORTPERIPH, PRCM_RUN_MODE_CLK);
// Configure PIN_55 for UART0 UART0_TX
MAP_PinTypeUART(PIN_55, PIN_MODE_3);
// Configure PIN_57 for UART0 UART0_RX
MAP_PinTypeUART(PIN_57, PIN_MODE_3);
// configuration, 8 bits length data width, 1 stop bit, no parity check
UARTConfigSetExpClk(UART_Handle->UART_PORT,PRCMPeripheralClockGet( UART_Handle->UART_PORTPERIPH),
UART_Handle->UART_BRATE, (UART_CONFIG_WLEN_8 | UART_CONFIG_STOP_ONE |
UART_CONFIG_PAR_NONE));
// disable UART since function above contained UARTenable
UARTDisable(UART_Handle->UART_PORT);
UARTIntEnable(UART_Handle->UART_PORT, UART_INT_TX | UART_INT_RX | UART_INT_RT); // enable interrupt for send and receive and receive timeout
UARTIntRegister(UART_Handle->UART_PORT, UART_ISR); //hook ISR
UARTFIFOEnable(UART_Handle->UART_PORT); //enable FIFO for send and receive
UARTFIFOLevelSet(UART_Handle->UART_PORT, UART_FIFO_TX1_8, UART_FIFO_RX4_8); //Interrupt occur when 7 bytes send from FIFO or read in FIFO
return UART_Handle;
}
I am dealing with a multi-thread bug for long time. After many test, I aware that the bug most possibly related to the UART FIFO transmitting.
Here is my description of my program:
The program is based on Free-RTOS. It has two threads running in parallel.
one thread is reading from sensor by I2C driver. another is sending data from the reading task to the computer by UART driver. I have a multi-task frame to transfer data from reading task to sending task. I have a timer file to control real time measuring. My I2C driver and UART driver are all interrupt based.
Problem description:
My data reading is good only for 2 mins. after that, my data shift. When do testing, I blocked the I2C driver, only put constant data "1 2 3 4 5 6 7 8" into the I2C read buffer, and transfer them to sending task to do UART send. And after 2 mins, my data read out will become "8 1 2 3 4 5 6 7" and after the first change, later changes happen quickly, "7 8 1 2 3 4 5 6" and continue till the end of timing.
this is plot of one channel of my data, begin with 8 and changing
I set many test to my program already, and I am sure my multi-task frame (trans data from reading task to sending task) don't alter the data.
Important observations:
1.when I put read and send into one thread, the data is very good, without this bug.
2.I set counting in my UART driver, found that the amount of sending is correct, here the sending is putting bytes into the TX FIFO. However, in the Excel, I read by a JAVA UART program, data missed. And JAVA UART program should be ok, since it works well when I use the single thread testing.
3. no memory leak error pump up by CCS while debugging.
So I am thinking, under the multi-thread environment, something stopped my UART TX FIFO transmit after I put bytes into that. But I can't find out where is wrong in my codes, they all seems correct.
And another reasonable reason can be something altered my pointer in the interrupt, in linked picture the "pCurrentWrite" pointer. But that pointer only accessed by UART driver, and before the interrupt finished, the next UART write can't enter.
I include partial code in the link, where I want to show that the data is put into the TX FIFO, and the send count is correct.
I can't include all codes, I don't think anyone want to check those long codes. and without running it, it's hard to find out where is wrong.
So, I just want to describe the situation i met, and see if someone had similar bug before. And maybe someone know the problem of UART FIFO under multi-thread environment. If someone who can help do want to see the code, we can do further contact.
thanks (My new account can't include picture yet, picture linked below)
This code is part of my UART driver, in the sending interrupt. I use the PutCharNonBlocking to put byte, since it return 1 if success, return 0 if FIFO full, so in this way I make sure the program will send all my data into the FIFO, which turn out to be true, all data was put into the TX FIFO
Although you have included the use of semaphores, I would suggest further reading on mutual exclusion and shared resources.
The solution is overly complex and it would be pertinent to better abstract your code and simplify.
Note: you should not cast the result of malloc.
UART_Handle = (UART_HANDLE)malloc(sizeof(UART_STATE));
I want to connect/bridge two serial ports in C.
I have 2 threads reading the ports, and writing to the other port.
Here is one example:
void *rfid_to_uart_thread(void *) {
char rfid_read_buffer[100];
int writeCounter;
do {
writeCounter = read(rfidCom, rfid_read_buffer, sizeof(rfid_read_buffer)-1);
if (writeCounter > 0) {
write(uartCom, rfid_read_buffer, writeCounter);
} else
usleep(25);
} while (!bKillBridgeThreads);
return NULL;}
The problem is, it seems that the writes are too slow. I often receive only half of the String on the other side. It seems like the write is asynchronously and thus the buffer is overwritten again in the next loop and overwrites the last 'write', so that the data is crippled?!
Is that right?
The ports are opened NON_BLOCKING and RW, Baudrate is and has to be 9600.
Looking at the man:
read() attempts to read up to count bytes from file descriptor fd into the buffer starting at buf.
In other wordsread does not grants to return all bytes send by other task, can give you a single byte up to sizeof(rfid_read_buffer)-1
What you can do is:
loop reading from rfidCom until the number of chars matches the number of chars sent.
You can use a specific terminator of messages and check for it to validate received message
encapsulate chars into a protocol message with an header that embed the message length, so the receiver can count the received chars and stop reading when last char is received.
For example:
void *rfid_to_uart_thread(void *)
{
char rfid_read_buffer[100] = {0};
int writeCounter;
char RXchar;
ssize_t retVal;
bool send = false;
do
{
memset(rfid_read_buffer, 0x00, sizeof(rfid_read_buffer));
send = true;
do
{
retVal = read(rfidCom, &RXchar, 1);
if (retVal > 0)
{
rfid_read_buffer[writeCounter] = RXchar;
writeCounter++;
}
else if (retVal < 0)
{
send = false;
RXchar = '\r'
break;
}
else
{
usleep(25);
}
}
while(RXchar != '\r');
if (send)
{
write(uartCom, rfid_read_buffer, writeCounter);
}
}
while (!bKillBridgeThreads);
return NULL;
}
OK, I've found a solution to my problem I think.
void *rfid_to_uart_thread(void *) {
char rfid_read_buffer[10];
ssize_t writeCounter = -1;
do {
writeCounter = read(rfidCom, &rfid_read_buffer, sizeof(rfid_read_buffer)-1);
if (writeCounter>0){
rfid_read_buffer[writeCounter] = 0;
LOGE("RFID -> UART: %s", rfid_read_buffer);
write(uartCom, rfid_read_buffer, writeCounter);
}else{
usleep(25);
}
tcdrain(uartCom);
} while (!bKillBridgeThreads);
return NULL;}
I've created my own define for a tcdrain, because the Android NDK I am using is not offering it in termios.h
Now, all the values seem to get transmitted to the UART port.
tcdrain is now defined as:
#define tcdrain(fd) ioctl(fd, TCSBRK, 1)