Im working with on pop3 clinet using openSSL library. However I run to this incosistency problem, when reading from BIO socket, 4 times out of 5 there is "\r\n.\r\n" at the end, indicating end of multi-line response from server. But sometimes only "\r\n" is being returned for some reason, and the rest ".\r\n" appears in the next BIO_read. Buffer is not overflowing, it is set to 1024 length and this happens even in 604 characters long response.
Example:
+OK 555 octets
Return-Path: test1#mail.local
Received: from [127.0.0.1] (mail.local [127.0.0.1])
by LAPTOP-R30GIL79 with ESMTPA
; Wed, 20 Oct 2021 11:55:24 +0200
Message-ID: <dad24da8-dd91-62ed-3338-95fe46890c32#mail.local>
Date: Wed, 20 Oct 2021 11:55:24 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
Thunderbird/91.2.0
Content-Language: sk
To: test1#mail.local
From: test1 <test1#mail.local>
Subject: testing
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Text testovacej spravy.
.
calling it exactly same after that results in:
+OK 555 octets
Return-Path: test1#mail.local
Received: from [127.0.0.1] (mail.local [127.0.0.1])
by LAPTOP-R30GIL79 with ESMTPA
; Wed, 20 Oct 2021 11:55:24 +0200
Message-ID: <dad24da8-dd91-62ed-3338-95fe46890c32#mail.local>
Date: Wed, 20 Oct 2021 11:55:24 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
Thunderbird/91.2.0
Content-Language: sk
To: test1#mail.local
From: test1 <test1#mail.local>
Subject: testing
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Text testovacej spravy.
\r\n
As you can see ".\r\n" is missing for some reason. Octets does match but it disappear somehow. There are no memory leaks. This is function Im using for reading BIO socket
int readBIO(){
memset(buf, 0, strlen(buf));
int x = BIO_read(bio,buf,BUFF_SIZE);
if (x == 0 )
{
fprintf(stderr, "Error: Trying to read from closed connection.\n");
freeAll();
exit(1);
}else if(x < 0){
if (! BIO_should_retry(bio))
{
fprintf(stderr, "Error: Reading from bio socked has failed.\n");
freeAll();
exit(1);
}
}
if (checkResponse() != 0)
{
fprintf(stderr, "Error: Incorrect command / typo in command detected.\n Or wrong username / password has been entered.\n Server responded with -ERR.\n");
freeAll();
exit(1);
}
return 0;
}
Anyone might have an idea why this is happening?
i am trying write a load test for this project https://github.com/theiterators/akka-http-microservice by using post request to server. but all the responses are failed. how can i resolve it
PostSimulation
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
class PostSimulation extends Simulation {
val httpProtocol = http
.baseUrl("http://localhost:9000") // Here is the root for all relative URLs
.inferHtmlResources()
.acceptHeader("*/*")
.acceptEncodingHeader("gzip, deflate")
.contentTypeHeader("application/json")
.userAgentHeader("PostmanRuntime/7.24.0")
val headers_0 = Map("Content-Type" -> "application/json")
val data = StringBody(
"""{"distance":662.3054387220024,
"ip1Info":{"query":"8.8.8.8",
"country":"United States",
"city":"Ashburn",
"lat":39.03,
"lon":-77.5},
"ip2Info":{"query":"93.184.216.34"
,"country":"United States",
"city":"Norwell",
"lat":42.1596,
"lon":-70.8217}}""")
val scn = scenario("PostSimulation")
.exec(http("request_1")
.post("/ip/8.8.8.8")
.headers(headers_0)
.body(data).asJson
.check(status.is(200)))
.exec(http("request_2")
.post("/ip/8.8.8.8")
.headers(headers_0)
.body(data).asJson
.check(status.is(200)))
.exec(http("request_3")
.post("/ip/8.8.8.8")
.headers(headers_0)
.body(data).asJson
.check(status.is(200)))
setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)
}
Error:
sbt:akka-http-microservice> gatling:testOnly PostSimulation
[info] compiling 1 Scala source to /home/mehveen/akka-http-microservice/target/scala-2.13/test-classes ...
io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:96)
at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:592)
at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:136)
at io.gatling.http.util.SslContextsFactory.<init>(SslContexts.scala:52)
at io.gatling.http.engine.HttpEngine$.apply(HttpEngine.scala:49)
at io.gatling.http.protocol.HttpProtocol$$anon$1.newComponents(HttpProtocol.scala:52)
at io.gatling.core.protocol.ProtocolComponentsRegistry.$anonfun$components$1(Protocol.scala:68)
at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:454)
at io.gatling.core.protocol.ProtocolComponentsRegistry.componentsFactory$1(Protocol.scala:68)
at io.gatling.core.protocol.ProtocolComponentsRegistry.comps$1(Protocol.scala:71)
at io.gatling.core.protocol.ProtocolComponentsRegistry.$anonfun$components$4(Protocol.scala:73)
at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:454)
at io.gatling.core.protocol.ProtocolComponentsRegistry.components(Protocol.scala:73)
at io.gatling.http.action.HttpActionBuilder.lookUpHttpComponents(HttpActionBuilder.scala:26)
at io.gatling.http.action.HttpRequestActionBuilder.build(HttpRequestActionBuilder.scala:33)
at io.gatling.core.structure.BuildAction.$anonfun$build$1(BuildAction.scala:28)
at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:169)
at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:165)
at scala.collection.immutable.List.foldLeft(List.scala:79)
at io.gatling.core.structure.BuildAction.build(BuildAction.scala:27)
at io.gatling.core.structure.BuildAction.build$(BuildAction.scala:26)
at io.gatling.core.structure.ScenarioBuilder.build(StructureBuilder.scala:51)
at io.gatling.core.structure.PopulationBuilder.build(PopulationBuilder.scala:100)
at io.gatling.core.scenario.SimulationParams.buildScenario(Simulation.scala:170)
at io.gatling.core.scenario.SimulationParams.$anonfun$scenarios$1(Simulation.scala:174)
at scala.collection.immutable.List.map(List.scala:246)
at io.gatling.core.scenario.SimulationParams.scenarios(Simulation.scala:174)
at io.gatling.app.Runner.run0(Runner.scala:82)
at io.gatling.app.Runner.run(Runner.scala:52)
at io.gatling.app.Gatling$.start(Gatling.scala:80)
at io.gatling.app.Gatling$.fromArgs(Gatling.scala:45)
at io.gatling.sbt.GatlingTask.liftedTree1$1(GatlingTask.scala:52)
at io.gatling.sbt.GatlingTask.execute(GatlingTask.scala:51)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
17:06:20.933 [pool-1-thread-1] DEBUG io.netty.util.internal.NativeLibraryLoader - netty_tcnative_linux_x86_64 cannot be loaded from java.library.path, now trying export to -Dio.netty.native.workdir: /tmp
java.lang.UnsatisfiedLinkError: no netty_tcnative_linux_x86_64 in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2660)
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:827)
at java.base/java.lang.System.loadLibrary(System.java:1871)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:351)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:96)
at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:592)
at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:136)
at io.gatling.http.util.SslContextsFactory.<init>(SslContexts.scala:52)
at io.gatling.http.engine.HttpEngine$.apply(HttpEngine.scala:49)
at io.gatling.http.protocol.HttpProtocol$$anon$1.newComponents(HttpProtocol.scala:52)
at io.gatling.core.protocol.ProtocolComponentsRegistry.$anonfun$components$1(Protocol.scala:68)
at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:454)
at io.gatling.core.protocol.ProtocolComponentsRegistry.componentsFactory$1(Protocol.scala:68)
at io.gatling.core.protocol.ProtocolComponentsRegistry.comps$1(Protocol.scala:71)
at io.gatling.core.protocol.ProtocolComponentsRegistry.$anonfun$components$4(Protocol.scala:73)
at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:454)
at io.gatling.core.protocol.ProtocolComponentsRegistry.components(Protocol.scala:73)
content: UnpooledHeapByteBuf(ridx: 0, widx: 265, cap: 265/265))
POST /ip/8.8.8.8 HTTP/1.1
accept: application/json
user-agent: PostmanRuntime/7.24.0
accept-encoding: gzip, deflate
host: localhost:9000
content-type: application/json
content-length: 265, content=null}
17:06:23.538 [gatling-1-2] DEBUG io.gatling.http.client.impl.HttpAppHandler - Read msg='DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 400 Bad Request
Server: akka-http/10.2.4
Date: Fri, 23 Apr 2021 12:06:23 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 166'
17:06:23.543 [gatling-1-2] DEBUG io.gatling.http.client.impl.HttpAppHandler - Read msg='DefaultLastHttpContent(data: PooledSlicedByteBuf(ridx: 0, widx: 166, cap: 166/166, unwrapped: PooledUnsafeDirectByteBuf(ridx: 319, widx: 319, cap: 2048)), decoderResult: success)'
17:06:23.563 [gatling-1-2] DEBUG io.gatling.http.engine.response.DefaultStatsProcessor - Request 'request_1' failed for user 1: status.find.is(200), but actually found 400
17:06:23.583 [gatling-1-2] DEBUG io.gatling.http.engine.response.DefaultStatsProcessor -
>>>>>>>>>>>>>>>>>>>>>>>>>>
Request:
request_1: KO status.find.is(200), but actually found 400
=========================
Session:
Session(PostSimulation,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts#6bb54726, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver#4971b2c, gatling.http.cache.baseUrl -> http://localhost:9000),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$474/0x00000001004fb040#7b4036c5,io.netty.channel.epoll.EpollEventLoop#4d79eb73)
=========================
HTTP request:
POST http://localhost:9000/ip/8.8.8.8
headers:
accept: application/json
user-agent: PostmanRuntime/7.24.0
accept-encoding: gzip, deflate
host: localhost:9000
content-type: application/json
content-length: 265
body:StringChunksRequestBody{contentType='application/json', charset=UTF-8, content={"distance":662.3054387220024,
"ip1Info":{"query":"8.8.8.8",
"country":"United States",
"city":"Ashburn",
"lat":39.03,
"lon":-77.5},
"ip2Info":{"query":"93.184.216.34"
,"country":"United States",
"city":"Norwell",
"lat":42.1596,
"lon":-70.8217}}}
=========================
HTTP response:
status:
400 Bad Request
headers:
Server: akka-http/10.2.4
Date: Fri, 23 Apr 2021 12:06:23 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 166
body:
The request content was malformed:
DecodingFailure at .ip1: Attempt to decode value on failed cursor
DecodingFailure at .ip2: Attempt to decode value on failed cursor
<<<<<<<<<<<<<<<<<<<<<<<<<
17:06:23.585 [gatling-1-2] DEBUG io.gatling.http.engine.tx.HttpTxExecutor - Sending request=request_2 uri=http://localhost:9000/ip/8.8.8.8: scenario=PostSimulation, userId=1
17:06:23.586 [gatling-1-2] DEBUG io.gatling.http.client.impl.HttpAppHandler - Write request WritableRequest{request=DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: UnpooledHeapByteBuf(ridx: 0, widx: 265, cap: 265/265))
POST /ip/8.8.8.8 HTTP/1.1
accept: application/json
user-agent: PostmanRuntime/7.24.0
accept-encoding: gzip, deflate
host: localhost:9000
content-type: application/json
content-length: 265, content=null}
17:06:23.592 [gatling-1-2] DEBUG io.gatling.http.client.impl.HttpAppHandler - Read msg='DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 400 Bad Request
Server: akka-http/10.2.4
Date: Fri, 23 Apr 2021 12:06:23 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 166'
17:06:23.593 [gatling-1-2] DEBUG io.gatling.http.client.impl.HttpAppHandler - Read msg='DefaultLastHttpContent(data: PooledSlicedByteBuf(ridx: 0, widx: 166, cap: 166/166, unwrapped: PooledUnsafeDirectByteBuf(ridx: 319, widx: 319, cap: 2048)), decoderResult: success)'
17:06:23.593 [gatling-1-2] DEBUG io.gatling.http.engine.response.DefaultStatsProcessor - Request 'request_2' failed for user 1: status.find.is(200), but actually found 400
17:06:23.595 [gatling-1-2] DEBUG io.gatling.http.engine.response.DefaultStatsProcessor -
>>>>>>>>>>>>>>>>>>>>>>>>>>
Request:
request_2: KO status.find.is(200), but actually found 400
=========================
Session:
Session(PostSimulation,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts#6bb54726, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver#4971b2c, gatling.http.cache.baseUrl -> http://localhost:9000),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$474/0x00000001004fb040#7b4036c5,io.netty.channel.epoll.EpollEventLoop#4d79eb73)
=========================
HTTP request:
POST http://localhost:9000/ip/8.8.8.8
headers:
accept: application/json
user-agent: PostmanRuntime/7.24.0
accept-encoding: gzip, deflate
host: localhost:9000
content-type: application/json
content-length: 265
body:StringChunksRequestBody{contentType='application/json', charset=UTF-8, content={"distance":662.3054387220024,
"ip1Info":{"query":"8.8.8.8",
"country":"United States",
"city":"Ashburn",
"lat":39.03,
"lon":-77.5},
"ip2Info":{"query":"93.184.216.34"
,"country":"United States",
"city":"Norwell",
"lat":42.1596,
"lon":-70.8217}}}
=========================
HTTP response:
status:
400 Bad Request
headers:
Server: akka-http/10.2.4
Date: Fri, 23 Apr 2021 12:06:23 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 166
body:
The request content was malformed:
DecodingFailure at .ip1: Attempt to decode value on failed cursor
DecodingFailure at .ip2: Attempt to decode value on failed cursor
<<<<<<<<<<<<<<<<<<<<<<<<<
17:06:23.597 [gatling-1-2] DEBUG io.gatling.http.engine.tx.HttpTxExecutor - Sending request=request_3 uri=http://localhost:9000/ip/8.8.8.8: scenario=PostSimulation, userId=1
17:06:23.598 [gatling-1-2] DEBUG io.gatling.http.client.impl.HttpAppHandler - Write request WritableRequest{request=DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: UnpooledHeapByteBuf(ridx: 0, widx: 265, cap: 265/265))
POST /ip/8.8.8.8 HTTP/1.1
accept: application/json
user-agent: PostmanRuntime/7.24.0
accept-encoding: gzip, deflate
host: localhost:9000
content-type: application/json
content-length: 265, content=null}
17:06:23.610 [gatling-1-2] DEBUG io.gatling.http.client.impl.HttpAppHandler - Read msg='DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 400 Bad Request
Server: akka-http/10.2.4
Date: Fri, 23 Apr 2021 12:06:23 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 166'
17:06:23.611 [gatling-1-2] DEBUG io.gatling.http.client.impl.HttpAppHandler - Read msg='DefaultLastHttpContent(data: PooledSlicedByteBuf(ridx: 0, widx: 166, cap: 166/166, unwrapped: PooledUnsafeDirectByteBuf(ridx: 319, widx: 319, cap: 1024)), decoderResult: success)'
17:06:23.612 [gatling-1-2] DEBUG io.gatling.http.engine.response.DefaultStatsProcessor - Request 'request_3' failed for user 1: status.find.is(200), but actually found 400
17:06:23.613 [gatling-1-2] DEBUG io.gatling.http.engine.response.DefaultStatsProcessor -
>>>>>>>>>>>>>>>>>>>>>>>>>>
Request:
request_3: KO status.find.is(200), but actually found 400
=========================
Session:
Session(PostSimulation,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts#6bb54726, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver#4971b2c, gatling.http.cache.baseUrl -> http://localhost:9000),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$474/0x00000001004fb040#7b4036c5,io.netty.channel.epoll.EpollEventLoop#4d79eb73)
=========================
HTTP request:
POST http://localhost:9000/ip/8.8.8.8
headers:
accept: application/json
user-agent: PostmanRuntime/7.24.0
accept-encoding: gzip, deflate
host: localhost:9000
content-type: application/json
content-length: 265
body:StringChunksRequestBody{contentType='application/json', charset=UTF-8, content={"distance":662.3054387220024,
"ip1Info":{"query":"8.8.8.8",
"country":"United States",
"city":"Ashburn",
"lat":39.03,
"lon":-77.5},
"ip2Info":{"query":"93.184.216.34"
,"country":"United States",
"city":"Norwell",
"lat":42.1596,
"lon":-70.8217}}}
=========================
HTTP response:
status:
400 Bad Request
headers:
Server: akka-http/10.2.4
Date: Fri, 23 Apr 2021 12:06:23 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 166
body:
The request content was malformed:
DecodingFailure at .ip1: Attempt to decode value on failed cursor
DecodingFailure at .ip2: Attempt to decode value on failed cursor
<<<<<<<<<<<<<<<<<<<<<<<<<
17:06:23.613 [gatling-1-2] DEBUG io.gatling.core.action.Exit - End user #1
17:06:23.622 [GatlingSystem-akka.actor.default-dispatcher-6] DEBUG io.gatling.core.controller.inject.Injector - End user #PostSimulation
17:06:23.623 [GatlingSystem-akka.actor.default-dispatcher-6] INFO io.gatling.core.controller.inject.Injector - All users of scenario PostSimulation are stopped
17:06:23.625 [GatlingSystem-akka.actor.default-dispatcher-6] INFO io.gatling.core.controller.inject.Injector - Stopping
17:06:23.626 [GatlingSystem-akka.actor.default-dispatcher-7] INFO io.gatling.core.controller.Controller - Injector has stopped, initiating graceful stop
================================================================================
2021-04-23 17:06:23 0s elapsed
---- Requests ------------------------------------------------------------------
> Global (OK=0 KO=3 )
> request_1 (OK=0 KO=1 )
> request_2 (OK=0 KO=1 )
> request_3 (OK=0 KO=1 )
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 400 3 (100.0%)
---- PostSimulation ------------------------------------------------------------
[##########################################################################]100%
waiting: 0 / active: 0 / done: 1
================================================================================
17:06:23.776 [GatlingSystem-akka.actor.default-dispatcher-4] DEBUG io.gatling.core.controller.Controller - StatsEngine was stopped
Simulation PostSimulation completed in 0 seconds
17:06:23.782 [gatling-1-2] DEBUG io.netty.buffer.PoolThreadCache - Freed 5 thread-local buffer(s) from thread: gatling-1-2
Parsing log file(s)...
17:06:23.855 [pool-1-thread-1] INFO io.gatling.charts.stats.LogFileReader - Collected List(/home/mehveen/akka-http-microservice/target/gatling/postsimulation-20210423120620892/simulation.log) from postsimulation-20210423120620892
17:06:23.866 [pool-1-thread-1] INFO io.gatling.charts.stats.LogFileReader - First pass
17:06:23.871 [pool-1-thread-1] INFO io.gatling.charts.stats.LogFileReader - First pass done: read 6 lines
17:06:23.882 [pool-1-thread-1] INFO io.gatling.charts.stats.LogFileReader - Second pass
17:06:23.917 [pool-1-thread-1] INFO io.gatling.charts.stats.LogFileReader - Second pass: read 6 lines
Parsing log file(s) done
Generating reports...
================================================================================
---- Global Information --------------------------------------------------------
> request count 3 (OK=0 KO=3 )
> min response time 7 (OK=- KO=7 )
> max response time 175 (OK=- KO=175 )
> mean response time 65 (OK=- KO=65 )
> std deviation 78 (OK=- KO=78 )
> response time 50th percentile 13 (OK=- KO=13 )
> response time 75th percentile 94 (OK=- KO=94 )
> response time 95th percentile 159 (OK=- KO=159 )
> response time 99th percentile 172 (OK=- KO=172 )
> mean requests/sec 3 (OK=- KO=3 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms 0 ( 0%)
> 800 ms < t < 1200 ms 0 ( 0%)
> t > 1200 ms 0 ( 0%)
> failed 3 (100%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 400 3 (100.0%)
================================================================================
Reports generated in 0s.
Please open the following file: /home/mehveen/akka-http-microservice/target/gatling/postsimulation-20210423120620892/index.html
[info] Simulation PostSimulation successful.
[info] Simulation(s) execution ended.
[success] Total time: 8 s, completed 23 Apr. 2021, 5:06:24 pm**
Your request body doesn't match what the server expects for this API.
The request content was malformed:
DecodingFailure at .ip1: Attempt to decode value on failed cursor
DecodingFailure at .ip2: Attempt to decode value on failed cursor
seems to indicate that you're supposed to have ip1 and ip2 objects in your payload.
=== edit
What you're sending with Gatling:
{
"distance":662.3054387220024,
"ip1Info":{
"query":"8.8.8.8",
"country":"United States",
"city":"Ashburn",
"lat":39.03,
"lon":-77.5
},
"ip2Info":{
"query":"93.184.216.34",
"country":"United States",
"city":"Norwell",
"lat":42.1596,
"lon":-70.8217
}
}
What you're sending with curl (-d option with the data wrapped in single quotes, I guess what comes after that is ignored):
{
"ip1":"8.8.8.8",
"ip2":"93.184.216.34"
}
Mail server: James.
Mail container: Maria DB
Mail parsing source is like below:
Return-Path: <monad#monad.com>
Delivered-To: yoonsang#bsecm.net
Received: from 192.168.10.159 ([192.168.10.159])
by WIN-55ERUE9ID5R (JAMES SMTP Server 2.3.2) with SMTP ID 374
for <monad#monad.net>;
Sat, 14 Nov 2015 16:08:06 +0900 (KST)
Received: from unknown (HELO cas01.bsecm.com) (192.168.10.14)
by 192.168.10.159 with ESMTP; 14 Nov 2015 16:07:48 +0900
X-Original-SENDERIP: 192.168.10.14
X-Original-MAILFROM: monad#monad.com
X-Original-RCPTTO: monad#monad.net
Resent-From: <monad#monad.com>
Received: from spam.bsecm.com (192.168.10.159) by cas01.bsecm.com
(192.168.10.14) with Microsoft SMTP Server (TLS) id 14.3.224.2; Sat, 14 Nov
2015 16:07:42 +0900
Received: from unknown (HELO ?219.255.136.51?) (219.255.136.51) by
192.168.10.159 with ESMTP; 14 Nov 2015 16:07:47 +0900
X-Original-SENDERIP: 219.255.136.51
X-Original-MAILFROM: allcredit#allcredit.co.kr
X-Original-RCPTTO: monad#monad.com
Date: Sat, 14 Nov 2015 16:07:56 +0900
Subject: =?euc-kr?B?W0tCxKu15Vdpc2VJbmZvIMfDt6+9ul3AscDnyKO01MDHIDExv/kgvcW/68Gkurizu7+qvK3A1LTPtNku?=
From: =?euc-kr?B?v8PFqbe5tfc=?= <allcredit#allcredit.co.kr>
To: <monad#monad.com>
Reply-To: <allcredit#allcredit.co.kr>
MIME-Version: 1.0
Content-Transfer-Encoding: BASE64
X-Mailer: Netpion Service Server v3.0.0
X-NetpionMsgID: 1447484876647.10284.1455.759978089,KB06,1006190838.1
Content-Type: multipart/mixed;
boundary="--Netpathy_Netpion.1447484876647.AA"
Message-ID: <11342c7a-220f-4538-9203-3610a6896f3e#CAS01.bsecm.com>
LS0tLU5ldHBhdGh5X05ldHBpb24uMTQ0NzQ4NDg3NjY0Ny5BQQ0KQ29udGVudC1UeXBlOiBtdWx0
aXBhcnQvYWx0ZXJuYXRpdmU7DQoJYm91bmRhcnk9Ii0tTmV0cGF0aHlfTmV0cGlvbi4xNDQ3NDg0
ODc2NjQ3LlpaIg0KDQotLS0tTmV0cGF0aHlfTmV0cGlvbi4xNDQ3NDg0ODc2NjQ3LlpaDQpDb250
NEthVzVwZENncE93bzgNCkwzTmpjbWx3ZEQ0PQ0KDQotLS0tTmV0cGF0aHlfTmV0cGlvbi4xNDQ3
NDg0ODc2NjQ3LkFBLS0NCg==
The contents is encoded by BASE64. At "Content-Transfer-Encoding: BASE64".
When I get contents Java tells me "Missing start boundary" exception.
byte[] messageBody = (byte[])("*FROM DB BLOB DATA*");
Message jamesMail = new MimeMessage(session, new ByteArrayInputStream(messageBody) );
Multipart mp = (Multipart)jamesMail.getContent();
int i = mp.getCount();
And now I decode mail contents.
This can parse mail contents to be readable.
java.io.InputStream is = jamesMail.getInputStream();
java.io.InputStream decodedIs = MimeUtility.decode(is, contentTransferEncoding);
mp = (Multipart)ms.getContent();
for(int m=0; m < mp.getCount(); m++){
// contents text processing
// attached file processing
}
But, mail contents has an attached file with encoded by BASE64.
And my code can not parse that attached file.
Question
-. How can I parsing pre-encoded mail contents using Javamail?
-. How can I parsing pre-encoded attachment file?
The message is incorrectly formatted. Multipart content MUST not be encoded, only "leaf" body parts may be encoded. "Netpion Service Server" is broken, please report the bug to the owner of that software.
You can work around this bug in the server by setting the "mail.mime.ignoremultipartencoding" System property to "false".
Temporary Solution)
I made a message [Header] + [ BASE64 Decoded Contents]
String headerLines = "";
String contentTransferEncoding = "";
while (headers.hasMoreElements()) {
Header h = (Header) headers.nextElement();
headerLines += String.format("%s: %s\r\n",h.getName(),h.getValue());
if(h.getName().equalsIgnoreCase("Content-Transfer-Encoding")){
contentTransferEncoding = (StringUtils.isEmpty(h.getValue()))?"":h.getValue();
}
}
InputStream hStream = new ByteArrayInputStream(headerLines.getBytes());
InputStream is = orgJamesMail.getInputStream();
InputStream decodedIs = MimeUtility.decode(is, contentTransferEncoding);
Message ms = new MimeMessage(session, mm);
mp = (Multipart)ms.getContent();
1. Get header information by String.
2. Convert header information of String to InputStream.
3. Concatenate "header inputstream" and "decoded contents inputstream"
4. And parse concatenated message stream.
Maybe this is need:
System.setProperty("mail.mime.multipart.ignoreexistingboundaryparameter", "false");
I have a little web server in C which works fine when it has to serve html or css files. But when it comes to non-text files (image, compressed files, etc.), the data are corrupted when received by the client, which can't display them.
The response from the server is like this :
HTTP/1.1 200 OK
Date: Wed, 11 Jun 2014 11:24:27 GMT
Server: MacFly/0.1.1
Cache-Control: max-age=60
Expires: Wed, 11 Jun 2014 11:25:27 GMT
Last-Modified: Wed, 11 Jun 2014 06:13:38 GMT
ETag: "1041463782"
Content-Length: 36851
Content-Type: image/jpeg
���
The last line correspond to the corrupted data.
To generate the data, I use the following function which read the demanded file (any format : html, css, jpg, etc.) and save it in file_contents :
void get_content(const char * filename, char * file_contents) {
FILE * file;
int file_size;
file_size = get_file_size(filename);
if (file = fopen(filename, "rb")) {
fread(file_contents, sizeof(char), file_size-1, file);
fclose(file);
}
}
i have Exchange2007. and
i am using the url:
"https://exchange2007.exchange.server.com/Exchange/username/calendar";
'calendar', or 'mailbox'( in your language! example, "boite de reception" in french or "calendário" in portuguese)
with that url that i'm using i can list my ressources, but can't send a mail or write an appointement! why?!?
See that i get a response of the server 207multistatus and ok, but the return a
HTTP/1.1 404 Resource Not Found
i wish a 201 created!!! (for my appointement)
someone have better ideia ?
thx.
Console:
>>>>>>> to server ---------------------------------------------------
PROPPATCH /Exchange/marcelo/calend%C3%A1rio HTTP/1.1
Authorization: Basic bWFyY2Vsb0BleGNoYW5nZTptdXN0YWZhMSQ=
Content-Type: text/xml; charset=utf-8
User-Agent: Jakarta Commons-HttpClient/2.0final
Host: exchange2007.exchange.snap.com.br
Content-Length: 1407
<D:propertyupdate xmlns:D="DAV:">
<D:set>
<D:prop>
<mapi xmlns="xmlns">
http://schemas.microsoft.com/mapi/
</mapi>
<Cmd xmlns="urn:">
saveappt
</Cmd>
<dtEnd xmlns="urn:schemas:calendar">
2009-06-30T10:30:00.000Z
</dtEnd>
<contentclass xmlns="DAV">
urn:content-classes:Appointment
</contentclass>
<Subject xmlns="urn:schemas:httpmail">
Changed Test Appointment Subject
</Subject>
<Location xmlns="urn:schemas:calendar">
do
</Location>
<responserequested xmlns="urn:schemas:calendar">
0
</responserequested>
<saveappt xmlns="urn:schemas:calendar:cmd">
1
</saveappt>
<ressource xmlns="DAV">
https://exchange2007.exchange.snap.com.br/Exchange/marcelo/calendárioassuntoteste.EML
</ressource>
<alldayevent xmlns="urn:schemas:calendar">
0
</alldayevent>
<to xmlns="urn:schemas:header">
adrien
</to>
<dtStart xmlns="urn:schemas:calendar">
2009-06-30T10:00:00.000Z
</dtStart>
<isfolder xmlns="DAV">
0
</isfolder>
<cmd xmlns="Cmd">
saveappt
</cmd>
<HtmlDescription xmlns="urn:schemas:httpmail">
Let's meet here
</HtmlDescription>
<outlookmessageclass xmlns="http://schemas.microsoft.com/exchange/subject-utf8=Appointment">
IPM.Appointement
</outlookmessageclass>
<instancetype xmlns="urn:schemas:calendar">
0
</instancetype>
<meetingstatus xmlns="urn:schemas:calendar">
CONFIRMED
</meetingstatus>
<finvited xmlns="urn:schemas:mapi">
0
</finvited>
<BusyType xmlns="urn:schemas:calendar">
BUSY
</BusyType>
</D:prop>
</D:set>
</D:propertyupdate>
------------------------------------------------------------------------
<<<<<<< from server ---------------------------------------------------
HTTP/1.1 207 Multi-Status
Date: Thu, 16 Jul 2009 20:29:40 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
MS-Exchange-Permanent-URL: https://exchange2007.exchange.snap.com.br/Exchange/marcelo/-FlatUrlSpace-/b3ee92320938254c828a96e2e269a417-a6271d
Repl-UID: <rid:b3ee92320938254c828a96e2e269a417000000a6282e>
Content-Type: text/xml
Content-Length: 825
ResourceTag: <rt:b3ee92320938254c828a96e2e269a417000000a6282eb3ee92320938254c828a96e2e269a41700545bb4844c>
MS-WebStorage: 08.01.10240
<a:multistatus
xmlns:a="DAV:"
xmlns:b="xmlns"
xmlns:c="urn:"
xmlns:d="urn:schemas:calendar"
xmlns:e="DAV"
xmlns:f="urn:schemas:httpmail"
xmlns:g="urn:schemas:calendar:cmd"
xmlns:h="urn:schemas:header"
xmlns:i="Cmd"
xmlns:j="http://schemas.microsoft.com/exchange/subject-utf8=Appointment"
xmlns:k="urn:schemas:mapi">
<a:response>
<a:href>
https://exchange2007.exchange.snap.com.br/Exchange/marcelo/Calend%C3%A1rio
</a:href>
<a:propstat>
<a:status>
HTTP/1.1 200 OK
</a:status>
<a:prop>
<b:mapi>
</b:mapi>
<c:Cmd>
</c:Cmd>
<d:dtEnd>
</d:dtEnd>
<e:contentclass>
</e:contentclass>
<f:Subject>
</f:Subject>
<d:Location>
</d:Location>
<d:responserequested>
</d:responserequested>
<g:saveappt>
</g:saveappt>
<e:ressource>
</e:ressource>
<d:alldayevent>
</d:alldayevent>
<h:to>
</h:to>
<d:dtStart>
</d:dtStart>
<e:isfolder>
</e:isfolder>
<i:cmd>
</i:cmd>
<f:HtmlDescription>
</f:HtmlDescription>
<j:outlookmessageclass>
</j:outlookmessageclass>
<d:instancetype>
</d:instancetype>
<d:meetingstatus>
</d:meetingstatus>
<k:finvited>
</k:finvited>
<d:BusyType>
</d:BusyType>
</a:prop>
</a:propstat>
</a:response>
</a:multistatus>
------------------------------------------------------------------------
>>>>>>> to server ---------------------------------------------------
PROPFIND /Exchange/marcelo/calend%C3%A1rio HTTP/1.1
Authorization: Basic bWFyY2Vsb0BleGNoYW5nZTptdXN0YWZhMSQ=
Content-Type: text/xml; charset=utf-8
User-Agent: Jakarta Commons-HttpClient/2.0final
Host: exchange2007.exchange.snap.com.br
Content-Length: 207
Depth: 0
<D:propfind xmlns:D="DAV:">
<D:prop>
<D:displayname>
</D:displayname>
<D:getcontentlength>
</D:getcontentlength>
<D:getcontenttype>
</D:getcontenttype>
<D:resourcetype>
</D:resourcetype>
<D:getlastmodified>
</D:getlastmodified>
<D:lockdiscovery>
</D:lockdiscovery>
</D:prop>
</D:propfind>
------------------------------------------------------------------------
<<<<<<< from server ---------------------------------------------------
HTTP/1.1 207 Multi-Status
Date: Thu, 16 Jul 2009 20:29:40 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Type: text/xml
Accept-Ranges: rows
MS-WebStorage: 08.01.10240
Transfer-Encoding: chunked
<a:multistatus
xmlns:a="DAV:"
xmlns:b="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/"
xmlns:c="xml:">
<a:response>
<a:href>
https://exchange2007.exchange.snap.com.br/Exchange/marcelo/Calend%C3%A1rio/
</a:href>
<a:propstat>
<a:status>
HTTP/1.1 200 OK
</a:status>
<a:prop>
<a:displayname>
Calendário
</a:displayname>
<a:getcontentlength b:dt="int">
0
</a:getcontentlength>
<a:resourcetype>
<a:collection>
</a:collection>
</a:resourcetype>
<a:getlastmodified b:dt="dateTime.tz">
2009-07-16T20:29:40.098Z
</a:getlastmodified>
<lockdiscovery xmlns="DAV:">
</lockdiscovery>
</a:prop>
</a:propstat>
<a:propstat>
<a:status>
HTTP/1.1 404 Resource Not Found
</a:status>
<a:prop>
<a:getcontenttype>
</a:getcontenttype>
</a:prop>
</a:propstat>
</a:response>
</a:multistatus>
I don't know anything about Exchange, but here are two WebDAV related comments.
1) Your property "mapi" in the namespace "xmlns" looks fishy. Not even Microsoft comes up with a namespace name that bad (I hope).
2) In the second server response, you are getting a status of 200 OK for most properties, except for getcontenttype (which is indeed 404).
Other than that: are you sure you it's the right thing to PROPPATCH against a calendar URL to do what you want?