Am using libcurl to communicate with Amazon S3.
GET calls are success whereas PUT (for uploading files) calls are failing with 403.
PUT call with same headers when ran through CURL-CLI are success.
I have disabled certificate checking. [[ curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0) ]]
Using CURLOPT_DEBUGFUNCTION option as specified in https://curl.haxx.se/libcurl/c/CURLOPT_DEBUGFUNCTION.html, am able to capture the following log.
=> Send header, 0000000578 bytes (0x00000242)
0000: PUT /M2000/activity/8c51f2240f9fc1e7d2329a24210e30c9_200_0603202
0040: 0065517_fota_dlready HTTP/1.1
005f: Host: fota.test.nvtl.s3.amazonaws.com
0086: Accept: */*
0093: Authorization : AWS4-HMAC-SHA256 Credential=AKIAJ2ZI2YKOFDBS4UMQ
00d3: /20200603/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-co
0113: ntent-sha256;x-amz-date, Signature=61613a8357e45ac6c14067856f1e1
0153: 56449adcba20d3b8d295ed1c67a126dda0c
0178: x-amz-content-sha256:e2b646fdb491ec4be82661a9ea86ce3b08fe5fabfda
01b8: 2b9a9f8c3d9a783135837
01cf: x-amz-date:20200603T065517Z
01ec: content-length: 3394
0202: content-type: application/octet-stream
022a: Expect: 100-continue
0240:
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: ....P
<= Recv header, 0000000024 bytes (0x00000018)
0000: HTTP/1.1 403 Forbidden
<= Recv header, 0000000036 bytes (0x00000024)
0000: x-amz-request-id: BE05183350F1A797
<= Recv header, 0000000090 bytes (0x0000005a)
0000: x-amz-id-2: b6rrLoMPXn0Umvv8YzFiVL8CX27oszT0mQqWnjxaBtD49DoqsI6C
0040: NedmUoOebVAf3R96Q7c59tg=
<= Recv header, 0000000031 bytes (0x0000001f)
0000: Content-Type: application/xml
<= Recv header, 0000000028 bytes (0x0000001c)
0000: Transfer-Encoding: chunked
<= Recv header, 0000000037 bytes (0x00000025)
0000: Date: Wed, 03 Jun 2020 06:55:19 GMT
<= Recv header, 0000000019 bytes (0x00000013)
0000: Connection: close
<= Recv header, 0000000018 bytes (0x00000012)
0000: Server: AmazonS3
<= Recv header, 0000000002 bytes (0x00000002)
0000:
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: ....0
<= Recv data, 0000000254 bytes (0x000000fe)
0000: f3
0004: <?xml version="1.0" encoding="UTF-8"?>.<Error><Code>AccessDenied
0044: </Code><Message>Access Denied</Message><RequestId>BE05183350F1A7
0084: 97</RequestId><HostId>b6rrLoMPXn0Umvv8YzFiVL8CX27oszT0mQqWnjxaBt
00c4: D49DoqsI6CNedmUoOebVAf3R96Q7c59tg=</HostId></Error>
00f9: 0
00fc:
=> Send SSL data, 0000000005 bytes (0x00000005)
0000: ....0
== Info: TLSv1.2 (OUT), TLS alert, Client hello (1):
=> Send SSL data, 0000000002 bytes (0x00000002)
0000: ..
== Info: TLSv1.2 (OUT), TLS header, Certificate Status (22):
=> Send SSL data, 0000000005 bytes (0x00000005)
0000: .....
== Info: TLSv1.2 (OUT), TLS handshake, Client hello (1):
=> Send SSL data, 0000000512 bytes (0x00000200)
0000: .........;r..f..g.]..50.=.+.xh..M..Ia.....0.,.(.$.............k.
0040: j.i.h.9.8.7.6.........2...*.&.......=.5.../.+.'.#.............g.
0080: #.?.>.3.2.1.0.........E.D.C.B.1.-.).%.......<./...A.............
00c0: ............3...$."...fota.test.nvtl.s3.amazonaws.com...........
0100: ................................ ...............................
0140: ......3t.........http/1.1.......................................
0180: ................................................................
01c0: ................................................................
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: ....W
== Info: TLSv1.2 (IN), TLS handshake, Server hello (2):
<= Recv SSL data, 0000000087 bytes (0x00000057)
0000: ...S...Z..J....v...6.F........i3..j:2. ..ptS.oT..h...8..J_..:o..
0040: )..a...................
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: ....T
== Info: TLSv1.2 (IN), TLS handshake, Certificate (11):
<= Recv SSL data, 0000002900 bytes (0x00000b54)
0000: ...P..M...0...0...........-........r..8..0...*.H........0d1.0...
0040: U....US1.0...U....DigiCert Inc1.0...U....www.digicert.com1#0!..U
0080: ....DigiCert Baltimore CA-2 G20...191109000000Z..210312120000Z0l
00c0: 1.0...U....US1.0...U....Washington1.0...U....Seattle1.0...U....A
0100: mazon.com, Inc.1.0...U....*.s3.amazonaws.com0.."0...*.H.........
0140: ....0.........[...)X.+\.x....<1j.....p.y.c..hx..c=.#..e......t,.
0180: g.^c3......z...}~>.2.eC.;...Y.;h..e....A..y....$#.K.#.&...2O..fD
01c0: ..7k..1.R..3....+~...{}.XT.gp4.l.0$.V.)Z.o.6..?..^Oz,%s....~...~
0200: ........z.c..P...~7G..-....!6K.....\....}....F.7..H.T.z..#p;..F
0240: #Jr...)L..xa..........0..|0...U.#..0......(thFg.p%t..E[.}\D0...U
0280: ........&.d...\.......'...0/..U...(0&..*.s3.amazonaws.com..s3.am
02c0: azonaws.com0...U...........0...U.%..0...+.........+.......0....U
0300: ...z0x0:.8.6.4http://crl3.digicert.com/DigiCertBaltimoreCA-2G2.c
0340: rl0:.8.6.4http://crl4.digicert.com/DigiCertBaltimoreCA-2G2.crl0L
0380: ..U. .E0C07..`.H...l..0*0(..+.........https://www.digicert.com/C
03c0: PS0...g.....0y..+........m0k0$..+.....0...http://ocsp.digicert.c
0400: om0C..+.....0..7http://cacerts.digicert.com/DigiCertBaltimoreCA-
0440: 2G2.crt0...U.......0.0..}..+.....y......m...i.g.v.......X......g
0480: p.<5.......w.........nMv;0.....G0E.!....)Y.!i...a.......4.w....w
04c0: .e..y. 3..cv.....R]....A.......a_.......v.D.e......#....(.......
0500: 1.?.3........nMv;(.....G0E.!...B).+.S..[..^..mD..&2.c8.*....7. B
0540: ..#.....f.s..B.....#mA&.u..v.C..u.......q...#...{G8W...R....d6..
0580: .....nMv;5.....F0D. ..._A/^..Y..V.....oJ.#.k.8...fDj. A..^..o...
05c0: .......09O.....dEgX.K.0...*.H.................x".V...6....2#....
0600: ..*.6q........CS.....-...=..ep...V......M.C......#.$.y.k.F?.v.&(
0640: .<N....a.o......i../..Q8K+^.;k86.F.....rW..._..>...3.q.Tc...l..
0680: $...fx0.;...)[.......f_....0T*....g.....<u{........8/6....."...J
06c0: .38...# %.K.nt.5M...'m5Pk..wH.0...Z....-.c..g0..c0..K...........
0700: ...&..;'....0...*.H........0Z1.0...U....IE1.0...U....Baltimore1.
0740: 0...U....CyberTrust1"0 ..U....Baltimore CyberTrust Root0...15120
0780: 8120507Z..250510120000Z0d1.0...U....US1.0...U....DigiCert Inc1.0
07c0: ...U....www.digicert.com1#0!..U....DigiCert Baltimore CA-2 G20..
0800: "0...*.H.............0..............s....\.u.\ps...z...#....?.!.
0840: .M..-...1..k..].Q..^r..f.....A..&....i.S..O>..:C+.Y.....Y#rZg...
0880: U..+............%.E......t....eP...E../..6......R....6q+.......[
08c0: [9...................Z../r....g....J. ...r....OB.M'BM.....u.6`.&
0900: T...7....)Y ..%........................0...0...U.........(thFg.p
0940: %t..E[.}\D0...U.#..0.....Y0.GX....T6.{:..M.0...U.......0.......0
0980: ...U...........04..+........(0&0$..+.....0...http://ocsp.digicer
09c0: t.com0:..U...3010/.-.+.)http://crl3.digicert.com/Omniroot2025.cr
0a00: l0=..U. .60402..U. .0*0(..+.........https://www.digicert.com/CPS
0a40: 0...*.H............./.7f....U.)..P.....(.t..i;D0=..I.h6..0....IB
0a80: cFR.i..I...W.....u....3.b.CT.c...S.........ex.3...>.p....x.3...,
0ac0: X..#.mA...../...pk.:.{....K..o..)....T.T.... .i,./?P...W...s..$.
0b00: ...E....).f...l.O.+.LG....A....FwB......%K.PW...N.?.%.A...bmo...
0b40: ......).......T.KI..
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: ....M
== Info: TLSv1.2 (IN), TLS handshake, Server key exchange (12):
<= Recv SSL data, 0000000333 bytes (0x0000014d)
0000: ...I...A.>...,R..3R.ry.../WTDH..b...lcB`.RB<dPQJ..2..$M.`6.A....
0040: ..2e.C.......$W+.8.g.].....A[q...,G.......u;.....q....8..:......
0080: G ....F.-.o.b54...P-.o..:|H..!......5.....O.7..d..El7.;....\....
00c0: t......'.v&...#..Y*.y..yoN....3 >...?...vH...i.J....K.....ar.:..
0100: ..Ll.............(Z..mx.;.....BQ...7.b.[...|..z.;..d....*. ..]M/
0140: ./.........=.
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: .....
== Info: TLSv1.2 (IN), TLS handshake, Server finished (14):
<= Recv SSL data, 0000000004 bytes (0x00000004)
0000: ....
=> Send SSL data, 0000000005 bytes (0x00000005)
0000: ....F
== Info: TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
=> Send SSL data, 0000000070 bytes (0x00000046)
0000: ...BA.wm.Jg'... .......,y8%B.....1..........}X.5.~....oC...T.].6
0040: .Q\..8
=> Send SSL data, 0000000005 bytes (0x00000005)
0000: .....
== Info: TLSv1.2 (OUT), TLS change cipher, Client hello (1):
=> Send SSL data, 0000000001 bytes (0x00000001)
0000: .
=> Send SSL data, 0000000005 bytes (0x00000005)
0000: ....#
== Info: TLSv1.2 (OUT), TLS handshake, Finished (20):
=> Send SSL data, 0000000016 bytes (0x00000010)
0000: ......1.e.].....
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: .....
== Info: TLSv1.2 (IN), TLS change cipher, Client hello (1):
<= Recv SSL data, 0000000001 bytes (0x00000001)
0000: .
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: ....#
== Info: TLSv1.2 (IN), TLS handshake, Finished (20):
<= Recv SSL data, 0000000016 bytes (0x00000010)
0000: ....t.....%. 1..
=> Send SSL data, 0000000005 bytes (0x00000005)
0000: ....P
When i ran the same from CLI it is success.
# curl --insecure -v -X PUT https://fota.test.nvtl.s3.amazonaws.c
om/M2000/activity/8c51f2240f9fc1e7d2329a24210e30c9_200_06032020065517_fota_dlrea
dy -H 'Authorization: AWS4-HMAC-SHA256 Credential=AKIAJ2ZI2YKOFDBS4UMQ/20200603/
us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, S
ignature=61613a8357e45ac6c14067856f1e156449adcba20d3b8d295ed1c67a126dda0c' -H x-
amz-content-sha256:e2b646fdb491ec4be82661a9ea86ce3b08fe5fabfda2b9a9f8c3d9a783135
837 -H x-amz-date:20200603T065517Z -H 'content-length: 3394' -H 'content-type: a
pplication/octet-stream' -T /opt/nvtl/data/8c51f2240f9fc1e7d2329a24210e30c9_
200_06032020065517_fota_dlready
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
> PUT /M2000/activity/8c51f2240f9fc1e7d2329a24210e30c9_200_06032020065517_fota_dlready HTTP/1.1
> Host: fota.test.nvtl.s3.amazonaws.com
> User-Agent: curl/7.52.1
> Accept: */*
> Authorization: AWS4-HMAC-SHA256 Credential=AKIAJ2ZI2YKOFDBS4UMQ/20200603/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=61613a8357e45ac6c14067856f1e156449adcba20d3b8d295ed1c67a126dda0c
> x-amz-content-sha256:e2b646fdb491ec4be82661a9ea86ce3b08fe5fabfda2b9a9f8c3d9a783135837
> x-amz-date:20200603T065517Z
> content-length: 3394
> content-type: application/octet-stream
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< x-amz-id-2: xnaNU1mmmuynkWQQDA0OJsgKKAYBOCRbx6lpTkj54HGEicXWOMtnd3qb4ZbY6vmogULiq7vOFcA=
< x-amz-request-id: BAE5F92ED54058EC
< Date: Wed, 03 Jun 2020 06:57:53 GMT
< ETag: "595f51bb7d2cc4c5c3f30b3bc3d350c3"
< Content-Length: 0
< Server: AmazonS3
<
Could any give hints what might actually be wrong here.
Is there any other method/way where we can have more verbose ouput regards the error.
Note:
HEADERS that are passed to both the requests (libcurl and curl_cli) are same.
Certificate check is disabled in both.
Response after adding user-agent.
0000: .....
=> Send header, 0000000609 bytes (0x00000261)
0000: PUT /M2000/activity/8c51f2240f9fc1e7d2329a24210e30c9_200_0603202
0040: 0142553_fota_dlready HTTP/1.1
005f: Host: fota.test.nvtl.s3.amazonaws.com
0086: User-Agent: libcurl-agent/1.0
00a5: Accept: */*
00b2: Authorization : AWS4-HMAC-SHA256 Credential=AKIAJ2ZI2YKOFDBS4UMQ
00f2: /20200603/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-co
0132: ntent-sha256;x-amz-date, Signature=9a72f22ce185e48c023a8d3314004
0172: ac2a76b91ccd56b208d769e27bafe7b1446
0197: x-amz-content-sha256:83a7ba5f8f0c829537965116840aad257f6b71e7899
01d7: bafc4c93542b681c9454a
01ee: x-amz-date:20200603T142554Z
020b: content-length: 3394
0221: content-type: application/octet-stream
0249: Expect: 100-continue
025f:
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: ....P
<= Recv header, 0000000024 bytes (0x00000018)
0000: HTTP/1.1 403 Forbidden
<= Recv header, 0000000036 bytes (0x00000024)
0000: x-amz-request-id: AD607FF3FA529447
<= Recv header, 0000000090 bytes (0x0000005a)
0000: x-amz-id-2: w0ZHrVJ9R8uslrB6kf9KLLRtshjksK9cLxeovH53GXL4uRoK17U6
0040: MJaWuhiGhZMKdIphBWkY+mE=
<= Recv header, 0000000031 bytes (0x0000001f)
0000: Content-Type: application/xml
<= Recv header, 0000000028 bytes (0x0000001c)
0000: Transfer-Encoding: chunked
<= Recv header, 0000000037 bytes (0x00000025)
0000: Date: Wed, 03 Jun 2020 14:25:57 GMT
<= Recv header, 0000000019 bytes (0x00000013)
0000: Connection: close
<= Recv header, 0000000018 bytes (0x00000012)
0000: Server: AmazonS3
<= Recv header, 0000000002 bytes (0x00000002)
0000:
<= Recv SSL data, 0000000005 bytes (0x00000005)
0000: ....0
<= Recv data, 0000000254 bytes (0x000000fe)
0000: f3
0004: <?xml version="1.0" encoding="UTF-8"?>.<Error><Code>AccessDenied
0044: </Code><Message>Access Denied</Message><RequestId>AD607FF3FA5294
0084: 47</RequestId><HostId>w0ZHrVJ9R8uslrB6kf9KLLRtshjksK9cLxeovH53GX
00c4: L4uRoK17U6MJaWuhiGhZMKdIphBWkY+mE=</HostId></Error>
UPDATED
Figured out the issue.
Issue seems to be with spacing.
"Authorization : AWS4-XXX" gives 403 error
whereas
"Authorization: AWS4-XXX" is success.
Thanks,
Trinadh
best guess: fota.test.nvtl.s3.amazonaws.com blocks requests without any User-Agent. several websites do this, for example Wikipedia.org does the same thing (not sure why),
curl the cli program has a default user-agent, it looks like User-Agent: curl/7.52.1, but libcurl doesn't have any default user-agent, you can set one with the CURLOPT_USERAGENT option.
Related
I'm trying to setup WireGuard VPN server on a cloud virtual server (Yandex cloud).
Server config:
[Interface]
Address = 10.128.0.19/24
MTU = 1500
SaveConfig = false
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t >
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables ->
ListenPort = 41820
PrivateKey = <cut>
[Peer]
PublicKey = 0fWTvnU+j4D4pXfv0hWtAJDatRj/DxgPH3zwrSbT7js=
AllowedIPs = 10.128.0.201/32
Client config:
[Interface]
PrivateKey = <cut>
Address = 10.128.0.200/32
DNS = 1.1.1.1, 1.0.0.1
[Peer]
PublicKey = g9HF8K1303CwDrYb0ga8/dBe8EY8tb3wlreO0lHA9iI=
AllowedIPs = 0.0.0.0/0
Endpoint = <cut>:41820
PersistentKeepalive = 25
I've enabled the net.ipv4.ip_forward=1 option on the server. The server is on the public cloud compute instance. The client is an Android device in a home network behind NAT.
When I turn on the tunnel, all the communications stops. I can't ping anything from the device. At the same time, I can see successful handshakes in the wg output:
interface: wg0
public key: g9HF8K1303CwDrYb0ga8/dBe8EY8tb3wlreO0lHA9iI=
private key: (hidden)
listening port: 41820
peer: 0fWTvnU+j4D4pXfv0hWtAJDatRj/DxgPH3zwrSbT7js=
endpoint: <cut>:38517
allowed ips: 10.128.0.201/32
latest handshake: 15 seconds ago
transfer: 2.25 KiB received, 124 B sent
I can't ping neither the VPN server internal IP address (10.128.0.19) nor any of public IPs (like 1.1.1.1).
The server's ifconfig output is the following:
$ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.128.0.19 netmask 255.255.255.0 broadcast 10.128.0.255
inet6 fe80::d20d:1bff:fe98:a801 prefixlen 64 scopeid 0x20<link>
ether d0:0d:1b:98:a8:01 txqueuelen 1000 (Ethernet)
RX packets 16530 bytes 2016056 (2.0 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12031 bytes 1483606 (1.4 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 177 bytes 14328 (14.3 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 177 bytes 14328 (14.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wg0: flags=209<UP,POINTOPOINT,RUNNING,NOARP> mtu 1500
inet 10.128.0.19 netmask 255.255.255.0 destination 10.128.0.19
unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 1000 (UNSPEC)
RX packets 145 bytes 16504 (16.5 KB)
RX errors 54 dropped 0 overruns 0 frame 54
TX packets 11 bytes 472 (472.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
The OS on the server is Ubuntu 20.04.
I tried to set MTU on the client side to 1500 but nothing changed.
What I'm doing wrong?
The issue was in the server's interface IP address. The correct one is
[Interface]
Address = 10.128.0.19/32
The subnet part should be 32 instead of 24 in my case.
After that, the connection works well.
The allowed ip is wrong in the server configuration.
Please change it from:
[Peer]
PublicKey = 0fWTvnU+j4D4pXfv0hWtAJDatRj/DxgPH3zwrSbT7js=
AllowedIPs = 10.128.0.201/32
to:
[Peer]
PublicKey = 0fWTvnU+j4D4pXfv0hWtAJDatRj/DxgPH3zwrSbT7js=
AllowedIPs = 10.128.0.200/32
I am trying to get about 1.4 to 2 MB of JSON data from a server using the curl library in C.
I have increased the Buffer size, connection timeout
curl_easy_setopt(conn->easy, CURLOPT_CONNECTTIMEOUT, 10);
curl_easy_setopt(conn->easy, CURLOPT_BUFFERSIZE, 524288);
#ifdef CURL_MAX_WRITE_SIZE
#undef CURL_MAX_WRITE_SIZE
#define CURL_MAX_WRITE_SIZE 524288
#endif
I did take a capture on the client I am running and the packets are not received completely.
I also tried the url from a browser and the browser is able to get the data and display.
Any help on this is appreciated.
Thanks
So basically my code is based on the example https://curl.haxx.se/libcurl/c/crawler.html
The difference here is that the timer_cb is called frequently and the easy_handle is released only if there is something to read.
static void timer_cb(int fd, short kind, void * userp)
{
GlobalInfo * g = (GlobalInfo *)userp;
CURLMcode rc;
(void)fd;
(void)kind;
rc = curl_multi_socket_action(g->multi,
CURL_SOCKET_TIMEOUT, 0, &g->still_running);
mcode_or_die("timer_cb: curl_multi_socket_action", rc);
check_multi_info(g);
}
I read the response here in this api and new_conn_init will create/malloc a new easy handle and call the curl_multi_add_handle.
static void check_multi_info(GlobalInfo * g)
{
char * eff_url;
CURLMsg * msg;
int msgs_left;
ConnInfo * conn = NULL;
CURL * easy;
.....
while ((msg = curl_multi_info_read(g->multi, &msgs_left)))
{
if (msg->msg == CURLMSG_DONE)
{
easy = msg->easy_handle;
res = msg->data.result;
do
{
rc = ERROR;
conn = NULL;
contentType = NULL;
iValidContent = 1;
..........
}
while (0);
u8 ret = 0;
curl_multi_remove_handle(g->multi, easy);
curl_easy_cleanup(easy);
if (conn)
new_conn_init(&(conn->reqInfo), g, 1);
}
new_conn_init ()
{
conn->easy = curl_easy_init();
curl_easy_setopt(conn->easy, CURLOPT_URL, conn->url);
curl_easy_setopt(conn->easy, CURLOPT_BUFFERSIZE, 524288);
curl_easy_setopt(conn->easy, CURLOPT_WRITEFUNCTION, write_cb);
curl_easy_setopt(conn->easy, CURLOPT_WRITEDATA, &(conn->respData));
// curl_easy_setopt(conn->easy, CURLOPT_HEADERDATA, &(conn->headerData));
// curl_easy_setopt(conn->easy, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(conn->easy, CURLOPT_VERBOSE, (long)shmHttpClientConfig->debugLibCurl);
curl_easy_setopt(conn->easy, CURLOPT_FORBID_REUSE, 1L);
curl_easy_setopt(conn->easy, CURLOPT_ERRORBUFFER, conn->error);
curl_easy_setopt(conn->easy, CURLOPT_PRIVATE, conn);
curl_easy_setopt(conn->easy, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
curl_easy_setopt(conn->easy, CURLOPT_DNS_CACHE_TIMEOUT, -1);
curl_easy_setopt(conn->easy, CURLOPT_NOPROGRESS, 1);
curl_easy_setopt(conn->easy, CURLOPT_CONNECTTIMEOUT, 10);
rc = curl_multi_add_handle(g->multi, conn->easy);
}
I did some debugging and looks like the connections are getting closed quite late in if you can see the op of the curl debug. I am sending url queries every 20 seconds.
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#1)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:37:53 GMT
<
* Found bundle for host 12.0.0.25: 0x9f8d20
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#2)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:38:13 GMT
<
* Found bundle for host 12.0.0.25: 0x9f8d20
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#3)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:38:33 GMT
<
* Found bundle for host 12.0.0.25: 0x9f8d20
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#4)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
* Operation timed out after 10000 milliseconds with 160728 out of 1526223 bytes received
* Closing connection 1
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:38:53 GMT
<
* Closing connection 4
* Found bundle for host 12.0.0.25: 0x9f8d20
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Operation timed out after 10001 milliseconds with 11584 out of 1526223 bytes received
* Closing connection 2
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#5)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:39:13 GMT
<
* Closing connection 5
* Operation timed out after 10001 milliseconds with 845632 out of 1526223 bytes received
* Closing connection 3
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#6)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:43:33 GMT
<
* Found bundle for host 12.0.0.25: 0x9f31f0
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#7)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< C
[root#BENU TEMP/MEG-1] ~# cat /opt/benu-data/admin/benu_libcurl_op.txt | more
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#1)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:37:53 GMT
<
* Found bundle for host 12.0.0.25: 0x9f8d20
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#2)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:38:13 GMT
<
* Found bundle for host 12.0.0.25: 0x9f8d20
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#3)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:38:33 GMT
<
* Found bundle for host 12.0.0.25: 0x9f8d20
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#4)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
* Operation timed out after 10000 milliseconds with 160728 out of 1526223 bytes received
* Closing connection 1
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 1526223
< Content-Type: application/json; charset=utf-8
< Date: Thu, 11 Apr 2019 11:38:53 GMT
<
* Closing connection 4
* Found bundle for host 12.0.0.25: 0x9f8d20
* Hostname 12.0.0.25 was found in DNS cache
* Trying 12.0.0.25...
* Hostname 12.0.0.29 was found in DNS cache
* Name '12.0.0.29' family 2 resolved to '12.0.0.29' family 2
* Local port: 0
* Operation timed out after 10001 milliseconds with 11584 out of 1526223 bytes received
* Closing connection 2
* Connected to 12.0.0.25 (12.0.0.25) port 1234 (#5)
> GET /xxxx HTTP/1.1
Host: 12.0.0.25:1234
Accept: */*
I'm trying to get a JSON object from an API and the devs for the API said they just enabled CORS but I'm stilling getting the bellow error.
XMLHttpRequest cannot load http://example.com/data/action/getGame/9788578457657. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://dev.our-domain.local' is therefore not allowed access.
I'm using AngularJS to get the JSON in a Service with
app.service("gameService", function ($http, $q)
{
function getGame(GameId) {
var deferred = $q.defer()
var url = 'http://example.com/data/action/getGame/' + gameId;
// var url = 'https://jsonplaceholder.typicode.com/albums/' + gameId; // THIS WORKS
$http({
method: 'GET',
cache: true,
url: url,
headers: {
'Content-Type': 'application/json;charset=UTF-8'
}
}).
then(function(response) {
//your code when success
deferred.resolve(response);
console.log('gameService HTTP CORS SUCCESS!');
}, function(response) {
//your code when fails
console.log('gameService HTTP CORS ERROR!');
// deferred.resolve('');
deferred.reject(response);
});
return deferred.promise;
}
this.getGame = getGame;
})
My AngularJS service works when I test it with jsonplaceholder which has CORS enabled.
Am I missing something?
The API devs said that two CORS-Headers are added to data.service responses but I don't see them. This is what I see on the headers when I curl down the JSON object.
$ curl -X HEAD -i http://example.com/data/action/getGame/9788578457657
HTTP/1.1 200 OK
Date: Wed, 14 Dec 2016 10:39:17 GMT
Server: WildFly/8
Expires: Wed, 14 Dec 2016 10:39:17 GMT
X-Powered-By: Undertow/1
X-dmg-elapsed-time: 20ms
X-dmg-host-address: 1??.??.???.??
Vary: Accept-Encoding,Origin
X-dmg-generated-time: Wed, 14 Dec 2016 10:39:17 GMT
Content-Type: application/json;charset=UTF-8
Content-Language: en-
X-dmg-node-name: defg_node_1
X-Varnish-Bereq-Backend: real_backend_foo_bar_uk
X-Varnish-Bereq-Retries: 0
Last-Modified: Wed, 14 Dec 2016 10:39:17 GMT
Cache-Control: public, max-age=300
X-Varnish: 6876870
Age: 0
Via: 1.1 varnish-v4
X-Varnish-Cache: MISS
X-Varnish-Trimen: www.trimen.com
X-Varnish-Served-By-Host: snarf.foo.uk
X-Varnish-Served-By-IP: 100.100.10.80
X-Varnish-Pool: http_pages
X-Varnish-Req-Backend-Hint: dead
X-Varnish-Req-Restarts: 0
X-Varnish-Hash: /data/action/getGame/9788578457657
X-Varnish-Backend-Ourself: varnish_server_snarf_foo_uk
X-DMG-Version: 6.20.51.2358
Accept-Ranges: none
Connection: keep-alive
Is this what I should be seeing with CORS enabled or is there something more?
Do I need to add more to my AngularJS Service to http get with Cors enabled, as add in more to:
headers: {
'Content-Type': 'application/json;charset=UTF-8'
}
UPDATE
Passing Origin: in the header on my curl request as suggested by #t.niese
$ curl -H "Origin: http://our-production-domain.com/" --verbose \
> http://example.com/data/action/getGame/9788578457657
* Trying 1?.???.??.???...
* Connected to http://example.com/ (1?.???.??.???) port 80 (#0)
> GET /data/action/getGame/9788578457657 HTTP/1.1
> Host: http://example.com/
> User-Agent: curl/7.43.0
> Accept: */*
> Origin: http://our-production-domain.com/
>
< HTTP/1.1 200 OK
< Date: Wed, 14 Dec 2016 11:05:24 GMT
< Server: WildFly/8
< Expires: Wed, 14 Dec 2016 11:05:24 GMT
< X-Powered-By: Undertow/1
< X-dmg-elapsed-time: 27ms
< X-dmg-host-address: 1??.??.???.??
< Vary: Accept-Encoding,Origin
< X-dmg-generated-time: Wed, 14 Dec 2016 11:05:24 GMT
< Content-Type: application/json;charset=UTF-8
< Content-Language: en-
< X-dmg-node-name: defg_node_1
< X-Varnish-Bereq-Backend: real_backend_foo_bar_uk
< X-Varnish-Bereq-Retries: 0
< Last-Modified: Wed, 14 Dec 2016 11:05:24 GMT
< Cache-Control: public, max-age=300
< X-Varnish: 6876870
< Age: 0
< Via: 1.1 varnish-v4
< X-Varnish-Cache: MISS
< X-Varnish-Trimen: www.trimen.com
< X-Varnish-Served-By-Host: snarf.foo.uk
< X-Varnish-Served-By-IP: 100.100.10.80
< X-Varnish-Pool: http_pages
< X-Varnish-Req-Backend-Hint: dead
< X-Varnish-Req-Restarts: 0
< X-Varnish-Hash: /data/action/getGame/9788578457657
< X-Varnish-Backend-Ourself: varnish_server_snarf_foo_uk
< X-DMG-Version: 6.20.51.2358
< Accept-Ranges: none
< Transfer-Encoding: chunked
< Connection: keep-alive
<
{
"errorMessage" : null,
"expiry" : "2016-12-14T11:05:24.379+0000",
"data" : {
// json object data here
}
* Connection #0 to host http://example.com/ left intact
}
and..
$ curl -H "Origin: http://qa.our-qa-domain.com/" --verbose \
> http://example.com/data/action/getGame/9788578457657
* Trying 1?.???.??.???...
* Connected to http://example.com/ (1?.???.??.???) port 80 (#0)
> GET /data/action/getGame/9788578457657 HTTP/1.1
> Host: http://example.com/
> User-Agent: curl/7.43.0
> Accept: */*
> Origin: http://qa.our-qa-domain.com/
>
< HTTP/1.1 200 OK
< Date: Wed, 14 Dec 2016 11:06:11 GMT
< Server: WildFly/8
< Expires: Wed, 14 Dec 2016 11:06:11 GMT
< X-Powered-By: Undertow/1
< X-dmg-elapsed-time: 18ms
< X-dmg-host-address: 1??.??.???.??
< Vary: Accept-Encoding,Origin
< X-dmg-generated-time: Wed, 14 Dec 2016 11:06:11 GMT
< Content-Type: application/json;charset=UTF-8
< Content-Language: en-
< X-dmg-node-name: defg_node_1
< X-Varnish-Bereq-Backend: real_backend_foo_bar_uk
< X-Varnish-Bereq-Retries: 0
< Last-Modified: Wed, 14 Dec 2016 11:06:11 GMT
< Cache-Control: public, max-age=300
< X-Varnish: 1343699
< Age: 0
< Via: 1.1 varnish-v4
< X-Varnish-Cache: MISS
< X-Varnish-Trimen: www.trimen.com
< X-Varnish-Served-By-Host: snarf.foo.uk
< X-Varnish-Served-By-IP: 100.100.10.80
< X-Varnish-Pool: http_pages
< X-Varnish-Req-Backend-Hint: dead
< X-Varnish-Req-Restarts: 0
< X-Varnish-Hash: /data/action/getGame/9788578457657
< X-Varnish-Backend-Ourself: varnish_server_snarf_foo_uk
< X-DMG-Version: 6.20.51.2358
< Accept-Ranges: none
< Content-Length: 2988
< Connection: keep-alive
<
{
"errorMessage" : null,
"expiry" : "2016-12-14T11:06:11.927+0000",
"data" : {
// json data object here
}
* Connection #0 to host http://example.com/ left intact
and..
$ curl -H "Origin: http://dev.my-dev.local/" --verbose \
> http://example.com/data/action/getGame/9788578457657
* Trying 1?.???.??.???...
* Connected to http://example.com/ (1?.???.??.???) port 80 (#0)
> GET /data/action/getGame/9788578457657 HTTP/1.1
> Host: http://example.com/
> User-Agent: curl/7.43.0
> Accept: */*
> Origin: http://dev.my-dev.local/
>
< HTTP/1.1 200 OK
< Date: Wed, 14 Dec 2016 11:07:10 GMT
< Server: WildFly/8
< Expires: Wed, 14 Dec 2016 11:07:10 GMT
< X-Powered-By: Undertow/1
< X-dmg-elapsed-time: 28ms
< X-dmg-host-address: 1??.??.???.??
< Vary: Accept-Encoding,Origin
< X-dmg-generated-time: Wed, 14 Dec 2016 11:07:10 GMT
< Content-Type: application/json;charset=UTF-8
< Content-Language: en-
< X-dmg-node-name: defg_node_1
< X-Varnish-Bereq-Backend: real_backend_foo_bar_uk
< X-Varnish-Bereq-Retries: 0
< Last-Modified: Wed, 14 Dec 2016 11:07:10 GMT
< Cache-Control: public, max-age=300
< X-Varnish: 6619151
< Age: 0
< Via: 1.1 varnish-v4
< X-Varnish-Cache: MISS
< X-Varnish-Trimen: www.trimen.com
< X-Varnish-Served-By-Host: snarf.foo.uk
< X-Varnish-Served-By-IP: 100.100.10.80
< X-Varnish-Pool: http_pages
< X-Varnish-Req-Backend-Hint: dead
< X-Varnish-Req-Restarts: 0
< X-Varnish-Hash: /data/action/getGame/9788578457657
< X-Varnish-Backend-Ourself: varnish_server_snarf_foo_uk
< X-DMG-Version: 6.20.51.2358
< Accept-Ranges: none
< Content-Length: 2988
< Connection: keep-alive
<
{
"errorMessage" : null,
"expiry" : "2016-12-14T11:07:10.764+0000",
"data" : {
// JSON object data here
}
* Connection #0 to host http://example.com/ left intact
}
2nd UPDATE
I disables same origin policy in Chrome and these are the headers to my JSON request from Chrome's network panel.
GET data/action/getGame/9788578457657 HTTP/1.1
Host: example.com
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Accept: application/json, text/plain, */*
Origin: http://dev.my-dev.local/
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36
Referer: http://dev.my-dev.local//game/id-9788578457657
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
HTTP/1.1 200 OK
Date: Wed, 14 Dec 2016 15:38:38 GMT
Server: WildFly/8
Expires: Wed, 14 Dec 2016 15:38:38 GMT
X-Powered-By: Undertow/1
X-dmg-elapsed-time: 25ms
X-dmg-host-address: 172.16.0.70
Vary: Accept-Encoding,Origin
X-dmg-generated-time: Wed, 14 Dec 2016 15:38:38 GMT
Content-Type: application/json;charset=UTF-8
Content-Language: en-
X-dmg-node-name: defg_node_1
Content-Encoding: gzip
Content-Length: 1109
X-Varnish-Bereq-Backend: real_backend_foo_bar_uk
X-Varnish-Bereq-Retries: 0
Last-Modified: Wed, 14 Dec 2016 15:38:38 GMT
Cache-Control: public, max-age=300
X-Varnish: 6619151
Age: 0
Via: 1.1 varnish-v4
X-Varnish-Cache: MISS
X-Varnish-Trimen: www.trimen.com
X-Varnish-Served-By-Host: snarf.foo.uk
X-Varnish-Served-By-IP: 100.100.10.80
X-Varnish-Pool: http_pages
X-Varnish-Req-Backend-Hint: dead
X-Varnish-Req-Restarts: 0
X-Varnish-Hash: /data/action/getGame/9788578457657
X-Varnish-Backend-Ourself: arnish_server_snarf_foo_uk
X-DMG-Version: 6.20.51.2358
Accept-Ranges: none
Connection: keep-alive
3rd UPDATE
So after changing the http method to OPTIONS as in
$http({
method: 'OPTIONS',
...
I ge this error in the chrome consoler
XMLHttpRequest cannot load http://example.com/data/action/getGame/9788578457657. Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://dev.my-dev.local'' is therefore not allowed access. The response had HTTP status code 405.
And these are the headers:
OPTIONS /data/action/getGame/9788578457657 HTTP/1.1
Host: example.com
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Access-Control-Request-Method: OPTIONS
Origin: http://dev.my-dev.local/
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36
Access-Control-Request-Headers:
Accept: */*
Referer: http://dev.my-dev.local//game/id-9788578457657
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
HTTP/1.1 405 Method Not Allowed
Date: Wed, 14 Dec 2016 16:52:03 GMT
Server: Varnish
X-Varnish: 6619151
X-Varnish-Trimen: www.trimen.com
X-Varnish-Served-By-Host: snarf.foo.uk
X-Varnish-Served-By-IP: 100.100.10.80
X-Varnish-Pool:
X-Varnish-Req-Backend-Hint: dead
X-Varnish-Req-Restarts: 0
X-DMG-Version: 6.20.51.2358
Content-Type: text/html; charset=utf-8
Retry-After: 5
Content-Length: 49669
Connection: keep-alive
You need to receive the following headers:
Access-Control-Allow-Origin: * (or whatever host you want to restrict to)
Access-Control-Allow-Methods: * (or whatever methods you want to restrict to)
Access-Control-Allow-Headers: Content-Type
Note the last one which is also important because you are setting Content-Type: application/json;charset=UTF-8. If you have any other custom headers you will need to add those too.
These are all to be done on the server though, your app doesn't need to do anything else.
Alternatively (if possible) you can opt to not use application/json at all and set your Content-Type to application/x-www-form-urlencoded, multipart/form-data, or text/plain and no preflight (OPTIONS) request will be done and it won't matter if CORS is enabled on the server or not.
The answer by ed inspired me on my solution
you need to send the following headers during a preflight request (OPTIONS method for the endpoint)
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET,HEAD,PUT,PATCH,POST,DELETE
Access-Control-Allow-Headers: Content-Type, Access-Control-Allow-Origin, xxx
where xxx is whatever additional headers you are sending when doing the POST/PUT/DELETE/etc request
note that when this solution works, I strongly suggest changing the * to restrict it to your known origins
I'm trying to send data using the booster pack cc3000 by Texas Instruments. Therefore I implemented a TCP server socket on my board. I can successfully accept a pending connection and send and receive data on the given socket. In my protocol the client is responsible for closing the connection after reading the response. But after some transmissions the transmissions becomes slow. If I inspect the WLAN traffic using Wireshark I see there is a problem with the socket close procedure. My client is a java based program. The board uses address 100 and the computer runs under 102.
The TCP stream looks the following:
31 4.696711000 192.168.2.102 192.168.2.100 TCP 66 50721 > http-alt [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1
32 4.700359000 192.168.2.100 192.168.2.102 TCP 58 http-alt > 50721 [SYN, ACK] Seq=0 Ack=1 Win=1460 Len=0 MSS=1460
33 4.700394000 192.168.2.102 192.168.2.100 TCP 54 50721 > http-alt [ACK] Seq=1 Ack=1 Win=17520 Len=0
34 4.700461000 192.168.2.102 192.168.2.100 HTTP 55 Continuation or non-HTTP traffic
35 4.705454000 192.168.2.100 192.168.2.102 TCP 54 http-alt > 50721 [ACK] Seq=1 Ack=2 Win=1460 Len=0
36 4.705476000 192.168.2.102 192.168.2.100 TCP 57 [TCP segment of a reassembled PDU]
37 4.709035000 192.168.2.100 192.168.2.102 TCP 54 http-alt > 50721 [ACK] Seq=1 Ack=5 Win=1460 Len=0
38 5.194961000 192.168.2.100 192.168.2.102 TCP 58 [TCP segment of a reassembled PDU]
39 5.196220000 192.168.2.100 192.168.2.102 HTTP 154 Continuation or non-HTTP traffic
40 5.196244000 192.168.2.102 192.168.2.100 TCP 54 50721 > http-alt [ACK] Seq=5 Ack=105 Win=17416 Len=0
41 5.196286000 192.168.2.102 192.168.2.100 TCP 54 50721 > http-alt [FIN, ACK] Seq=5 Ack=105 Win=17416 Len=0
42 5.202194000 192.168.2.100 192.168.2.102 TCP 54 http-alt > 50721 [ACK] Seq=105 Ack=6 Win=1460 Len=0
138 24.245036000 192.168.2.100 192.168.2.102 TCP 54 http-alt > 50721 [FIN, ACK] Seq=105 Ack=6 Win=1460 Len=0
139 24.245060000 192.168.2.102 192.168.2.100 TCP 54 50721 > http-alt [ACK] Seq=6 Ack=106 Win=17416 Len=0
After about 10 transmissions I get a FIN/ACK cascade. The last FIN/ACK above is part of this cascade. It looks like if the sockets are not completely closed and the HW module starts now closing all sockets in a row.
My Java - client does the following
Socket b = new Socket("192.168.2.100",8080);
OutputStream o = b.getOutputStream();
o.write(10);
o.write(0);
o.write(0);
o.write(0);
o.flush();
InputStream i = b.getInputStream();
int id = i.read();
int gId = i.read();
int lengthA = i.read();
int lengthB = i.read();
int length = (lengthB<<8)|lengthA;
if(length < 0|| length > 1000)
{
b.close();
return;
}
System.out.println(new Date()+" GantryID: "+gId+" Package with id: "+id+" has length: "+length+" payload: ");
DataInputStream ds = new DataInputStream(i);
byte[] buffer = new byte[length];
ds.readFully(buffer);
b.close();
System.out.println(new String(buffer));
The server is a little more complex but the important commands are:
return recv(handle, data, size, 0); //Read request header
return recv(handle, data, size, 0); //Read request payload
send(handle, data, size, 0); //Write response header
send(handle, data, size, 0); //Write response payload
//No Close only set socket handle to -1
Does anyone have a idea what is going on. I would appreciate any help and ideas.
I'm writing a proxy in C in Linux 3.0 kernel. The proxy forwards the request from browser to the target server and read response from server.
This is one request:
GET http://www.yongchuan.org/ HTTP/1.1
Host: www.yongchuan.org
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Proxy-Connection: keep-alive
This is one response the proxy read:
HTTP/1.1 200 OK
Content-Length: 3170
Content-Type: text/html
Content-Location: http://www.yongchuan.org/index.html
Last-Modified: Fri, 22 Jul 2011 01:28:50 GMT
Accept-Ranges: bytes
ETag: "72d9d7b5e48cc1:1a73"
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Sun, 04 Mar 2012 16:26:05 GMT
<html>
<head>
..............
<tr>
<td width="43%" ></td>
<td width="46%"></td>
</tr>
<tr>
<td width="43%"><img src="images/triangle.gif"> ��ѧ�ڿγ̣�Ӧ��ͼ��</td>
<td width="46%" ><img src=
The problem is, the actual size of bytes the proxy read is 2880 and the data ends up suddenly, much less than Context length: 3170.
My code is as below:
241 int readlen;
242 char buffer[128 * 4096];
243 do {
244 readlen = read(servfd, buffer, 128 * 4096);
245 printf("readlen:%d\n", readlen);
246 if( readlen < 0 ) {
247 perror("read() from server failed");
248 }
249 printf("read content:\n%s", buffer2);
250 write(cliefd, buffer, readlen);
251 } while( readlen == 128 * 4096 );
Here servfd is the socket to server, cliefd is the socket to client.
What's the possible reason? Thanks!
while( readlen == 128 * 4096 );
The while condition is wrong. You should be checking readlen > 0 instead.
There's no requirement that read only hands you completely filled buffers. In the worst case (highly unlikely with current TCP stacks), it could hand you the bytes one by one.
At any rate, if the server doesn't close the connection after delivering the output you must keep reading until you get the Content-Length you were promised.