Why is my .Net Core 3.1 API/SPA stopping suddenly when I use file-saver on the client app while running in Visual Studio/IIS? - reactjs

I have an odd issue related to running a React front end with a .Net Core 3.1 Web API. I used the Visual Studio template to create the app, so the client app exists inside my .Net Core project and is launched when I launch the API.
I have an API endpoint that returns a text/csv file via a FileStreamResult, and then I am using the npm package file-saver to allow the client to save the file on the front end. The file is being returned and saved successfully, but every time I run the saveAs function from my client app, the API shuts down when running in IIS on Visual Studio It just stops. Nothing in the logs, though I have set them to verbose. It shows the API being hit and the response being returned, but nothing else. I've also searched the Event Logs, but can't find any errors at the times of the shutdowns. It happens consistently. If I dont' save the file, the API does not shut down, but as soon as I save the file, it does.
If I run the API from Powershell using 'dotnet run,' the file is returned and saved successfully and the API does not shut down. I feel like this might be an IIS configuration issue, but I'm not sure how to resolve it.
I'm running Visual Studio Community 2019 version 16.10.2 and .Net Framework version 4.8.04084
The code in my Startup.cs that runs the React app is
app.UseSpa(spa =>
{ spa.Options.SourcePath = "ClientApp";
if (env.IsDevelopment())
{
spa.UseProxyToSpaDevelopmentServer("http://localhost:3000");
}
});
and my front-end code to retrieve and save the file looks like
try {
const response = await fetch(URL, options);
const file = await response.blob();
saveAs(file, fileName);
} catch (err) {
...
}
And my logs:
{"Timestamp":"2021-07-22T08:50:58.9565584-05:00","Level":"Information","MessageTemplate":"Executing endpoint '{EndpointName}'","RenderedMessage":"Executing endpoint '\"CommissionDatabase.WebApp.Controllers.ExportController.GetAICommissionExport (CommissionDatabase.WebApp)\"'","Properties":{"EndpointName":"CommissionDatabase.WebApp.Controllers.ExportController.GetAICommissionExport (CommissionDatabase.WebApp)","EventId":{"Name":"ExecutingEndpoint"},"SourceContext":"Microsoft.AspNetCore.Routing.EndpointMiddleware","RequestId":"8000008b-0004-fd00-b63f-84710c7967bb","RequestPath":"/api/exports/aicommissions","SpanId":"|db35c5e7-4e2963d2e6602a10.","TraceId":"db35c5e7-4e2963d2e6602a10","ParentId":"","Application":"ims-api"}}
{"Timestamp":"2021-07-22T08:50:58.9619447-05:00","Level":"Information","MessageTemplate":"Route matched with {RouteData}. Executing controller action with signature {MethodInfo} on controller {Controller} ({AssemblyName}).","RenderedMessage":"Route matched with \"{action = \\\"GetAICommissionExport\\\", controller = \\\"Export\\\"}\". Executing controller action with signature \"System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] GetAICommissionExport(CommissionDatabase.Domain.DTO.FileExportDTO)\" on controller \"CommissionDatabase.WebApp.Controllers.ExportController\" (\"CommissionDatabase.WebApp\").","Properties":{"RouteData":"{action = \"GetAICommissionExport\", controller = \"Export\"}","MethodInfo":"System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] GetAICommissionExport(CommissionDatabase.Domain.DTO.FileExportDTO)","Controller":"CommissionDatabase.WebApp.Controllers.ExportController","AssemblyName":"CommissionDatabase.WebApp","EventId":{"Id":3,"Name":"ControllerActionExecuting"},"SourceContext":"Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker","ActionId":"1197f939-11ae-4c34-bd03-922af3942f00","ActionName":"CommissionDatabase.WebApp.Controllers.ExportController.GetAICommissionExport (CommissionDatabase.WebApp)","RequestId":"8000008b-0004-fd00-b63f-84710c7967bb","RequestPath":"/api/exports/aicommissions","SpanId":"|db35c5e7-4e2963d2e6602a10.","TraceId":"db35c5e7-4e2963d2e6602a10","ParentId":"","Application":"ims-api"}}
{"Timestamp":"2021-07-22T08:50:59.0250952-05:00","Level":"Information","MessageTemplate":"{HostingRequestFinishedLog:l}","RenderedMessage":"Request finished in 2043.6825ms 304 ","Properties":{"ElapsedMilliseconds":2043.6825,"StatusCode":304,"ContentType":null,"HostingRequestFinishedLog":"Request finished in 2043.6825ms 304 ","EventId":{"Id":2},"SourceContext":"Microsoft.AspNetCore.Hosting.Diagnostics","RequestId":"800001f8-0002-fb00-b63f-84710c7967bb","RequestPath":"/favicon.ico","SpanId":"|db35c5e6-4e2963d2e6602a10.","TraceId":"db35c5e6-4e2963d2e6602a10","ParentId":"","Application":"ims-api"},"Renderings":{"HostingRequestFinishedLog":[{"Format":"l","Rendering":"Request finished in 2043.6825ms 304 "}]}}
{"Timestamp":"2021-07-22T08:50:59.7133458-05:00","Level":"Information","MessageTemplate":"Entity Framework Core {version} initialized '{contextType}' using provider '{provider}' with options: {options}","RenderedMessage":"Entity Framework Core \"3.1.16\" initialized '\"CommissionDatabaseContext\"' using provider '\"Microsoft.EntityFrameworkCore.SqlServer\"' with options: \"None\"","Properties":{"version":"3.1.16","contextType":"CommissionDatabaseContext","provider":"Microsoft.EntityFrameworkCore.SqlServer","options":"None","EventId":{"Id":10403,"Name":"Microsoft.EntityFrameworkCore.Infrastructure.ContextInitialized"},"SourceContext":"Microsoft.EntityFrameworkCore.Infrastructure","ActionId":"1197f939-11ae-4c34-bd03-922af3942f00","ActionName":"CommissionDatabase.WebApp.Controllers.ExportController.GetAICommissionExport (CommissionDatabase.WebApp)","RequestId":"8000008b-0004-fd00-b63f-84710c7967bb","RequestPath":"/api/exports/aicommissions","SpanId":"|db35c5e7-4e2963d2e6602a10.","TraceId":"db35c5e7-4e2963d2e6602a10","ParentId":"","Application":"ims-api"}}
{"Timestamp":"2021-07-22T08:50:59.9887399-05:00","Level":"Information","MessageTemplate":"Executed DbCommand ({elapsed}ms) [Parameters=[{parameters}], CommandType='{commandType}', CommandTimeout='{commandTimeout}']{newLine}{commandText}","RenderedMessage":"Executed DbCommand (\"23\"ms) [Parameters=[\"#__p_0='?' (DbType = Int32)\"], CommandType='Text', CommandTimeout='30']\"\r\n\"\"SELECT TOP(1) [c].[Id], [c].[Name]\r\nFROM [Carriers] AS [c]\r\nWHERE [c].[Id] = #__p_0\"","Properties":{"elapsed":"23","parameters":"#__p_0='?' (DbType = Int32)","commandType":"Text","commandTimeout":30,"newLine":"\r\n","commandText":"SELECT TOP(1) [c].[Id], [c].[Name]\r\nFROM [Carriers] AS [c]\r\nWHERE [c].[Id] = #__p_0","EventId":{"Id":20101,"Name":"Microsoft.EntityFrameworkCore.Database.Command.CommandExecuted"},"SourceContext":"Microsoft.EntityFrameworkCore.Database.Command","ActionId":"1197f939-11ae-4c34-bd03-922af3942f00","ActionName":"CommissionDatabase.WebApp.Controllers.ExportController.GetAICommissionExport (CommissionDatabase.WebApp)","RequestId":"8000008b-0004-fd00-b63f-84710c7967bb","RequestPath":"/api/exports/aicommissions","SpanId":"|db35c5e7-4e2963d2e6602a10.","TraceId":"db35c5e7-4e2963d2e6602a10","ParentId":"","Application":"ims-api"}}
{"Timestamp":"2021-07-22T08:51:00.0737948-05:00","Level":"Information","MessageTemplate":"Executing {FileResultType}, sending file with download name '{FileDownloadName}' ...","RenderedMessage":"Executing \"Microsoft.AspNetCore.Mvc.FileStreamResult\", sending file with download name '\"\"' ...","Properties":{"FileResultType":"Microsoft.AspNetCore.Mvc.FileStreamResult","FileDownloadName":"","EventId":{"Id":2,"Name":"ExecutingFileResultWithNoFileName"},"SourceContext":"Microsoft.AspNetCore.Mvc.Infrastructure.FileStreamResultExecutor","ActionId":"1197f939-11ae-4c34-bd03-922af3942f00","ActionName":"CommissionDatabase.WebApp.Controllers.ExportController.GetAICommissionExport (CommissionDatabase.WebApp)","RequestId":"8000008b-0004-fd00-b63f-84710c7967bb","RequestPath":"/api/exports/aicommissions","SpanId":"|db35c5e7-4e2963d2e6602a10.","TraceId":"db35c5e7-4e2963d2e6602a10","ParentId":"","Application":"ims-api"}}
{"Timestamp":"2021-07-22T08:51:00.0777075-05:00","Level":"Information","MessageTemplate":"Executed action {ActionName} in {ElapsedMilliseconds}ms","RenderedMessage":"Executed action \"CommissionDatabase.WebApp.Controllers.ExportController.GetAICommissionExport (CommissionDatabase.WebApp)\" in 1115.5799ms","Properties":{"ActionName":"CommissionDatabase.WebApp.Controllers.ExportController.GetAICommissionExport (CommissionDatabase.WebApp)","ElapsedMilliseconds":1115.5799,"EventId":{"Id":2,"Name":"ActionExecuted"},"SourceContext":"Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker","ActionId":"1197f939-11ae-4c34-bd03-922af3942f00","RequestId":"8000008b-0004-fd00-b63f-84710c7967bb","RequestPath":"/api/exports/aicommissions","SpanId":"|db35c5e7-4e2963d2e6602a10.","TraceId":"db35c5e7-4e2963d2e6602a10","ParentId":"","Application":"ims-api"}}
Thanks!

Related

Flink Rest API : /jars/upload returning 404

Following is my code snippet used for uploading Jar in Flink. I am getting 404 response for this post request. Following is the output for request. I also tried updating the url with /v1/jars/upload but same response. All the API related to jars is giving me same response. I am running this code inside AWS lambda which is present in same vpc where EMR exists which is runing my Flink Job. APIs like /config, /jobs working in this lambda, only APIs like upload jar, submit jobs not working and getting 404 for them
<Response [404]> {"errors":["Not found: /jars/upload"]}
Also tried the same thing by directly logging into job manager node and running curl command, but got the same response. I am using Flink 1.14.2 version on EMR cluster
curl -X POST -H "Expect:" -F
"jarfile=#/home/hadoop/test-1.0-global-14-dyn.jar"
http://ip-10-0-1-xxx:8081/jars/upload
{"errors":["Not found:> /jars/upload"]}
import json
import requests
import boto3
import os
def lambda_handler(event, context):
config = dict(
service_name="s3",
region_name="us-east-1"
)
s3_ = boto3.resource(**config)
bucket = "dv-stream-processor-na-gamma"
prefix = ""
file = "Test-1.0-global-14-dyn.jar"
path = "/tmp/"+file;
try:
s3_.Bucket(bucket).download_file(f"{file}", "/tmp/"+file)
except botocore.exceptions.ClientError as e:
print(e.response['Error']['Code'])
if e.response['Error']['Code'] == "404":
print("The object does not exist.")
print(os.path.isfile('/tmp/' + file))
response = requests.post(
"http://ip-10-0-1-xx.ec2.internal:8081/jars/upload",
files={
"jarfile": (
os.path.basename(path),
open(path, "rb"),
"application/x-java-archive"
)
}
)
print(response)
print(response.text)
Reason for upload jar was not working for me was I was using Flink "Per Job" cluster mode where it was not allowed to submit job via REST API. I updated the cluster mode to "Session" mode and it started working
References for Flink cluster mode information :
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/overview/
Code you can refer to start cluster in session mode : https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#starting-a-flink-session-on-yarn

How to use pytessseract on remote server?

Problem-
I am using a simple pytesseract + opencv implementation in my discord bot but tesseract is only working when i host it locally. On hosting the file on a remote server (In my case contabo). It shows the following error.
Error-
Extension 'cogs.Wlping' raised an error: ModuleNotFoundError: no module named 'pytesseract'
Code-
async def on_message(msg):
img = cv.imread("test.png")
result = pytesseract.image_to_string(img, lang='eng', config='--psm 11')
await msg.channel.send(result)
I even added tesseract and pytesseract in the requirements.txt file on server. If there is anything else I should do please guide me through.

Maven cxf-codegen-plugin with NTLM Authentication - fails with java.io.IOException: Server returned HTTP response code: 401 for URL

I'm trying to generate java code using cxf-codegen-plugin for a SOAP webservice that is secured over NTLM. I'm not sure how do pass on the username/pwd/domain information to maven to connect to the remote server while generating the java code. I could open the wsdl using the browser using the username/pwd/domain combination I have. However, I'm wondering how to do it via command-line by passing some extra vm args to - mvn package. Or is there any other plugin config that I can do which helps me to do this?

How to disable loading external urls on seleniumlibrary/robotframework

I started playing with Seleniumlibrary tests (run with robotframework) and as our websites have ads and metrics and so on, every time I run a test those URLs get loaded.
Is there a way to tell selenium/robotframework to not load certain types of URLs or prevent it to load external resources (i.e. everything that is not from localhost).
You can do that with browsermob-proxy. Once you have that installed, you simply setup the proxy and set the blacklist.
ProxyServer server = new ProxyServer(9000)
server.start();
final DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, server.seleniumProxy());
//Send a 200 for all requests to the facebook cdn
server.blacklistRequests("http://.*\\.fbcdn.net/.*", 200);
//Finish setting up your driver
WebDriver driver = new SomeDriverImpl(capabilities);
I believe the following will work with this python wrapper (the regex might be slightly different):
from browsermobproxy import Server
server = Server("path/to/browsermob-proxy")
server.start()
proxy = server.create_proxy()
proxy.blacklist('http://.*\\.fbcdn.net/.*', 200)
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
...
proxy.stop()
driver.quit()
This can now be done in two lines in Chrome:
driver.execute_cdp_cmd('Network.setBlockedURLs', {"urls": ["www.baidu.com"]})
driver.execute_cdp_cmd('Network.enable', {})

Unit testing OAuth JS with Mocha

I'm working on a JS based project that runs off GAE and part of the code gets the user's avatar using OAuth from Facebook, Twitter or Google. I'm trying to write tests in Mocha in order to test this but I'm running into some problems.
The code works when I test it in the front end, and the way I envisaged it to work would be to use ZombieJS to run the app on GAE's dev_appserver.py, fire the OAuth functions, fill in the appropriate auth stuff and then complete the test by returning the image URL.
However the first hurdle I've got is that it appears that NodeJS's server is not allowing GAE's server to run on the same IP address. For example:
exec 'dev_appserver.py .', ->
console.log arguments
This returns the error 'Address already in use'. How can I get around this apart from running it on a different machine? Is it possible to tell NodeJS to not reserve the whole IP and just a port? I'm running GAE on 8080 and it works fine when it isn't invoked by NodeJS.
The second problem is ZombieJS. I'm trying to figure out a way I can listen to when new windows are opened and, essentially, tail the console of the browser. I've started two discussions on the Google group but no one has responded yet (https://groups.google.com/forum/?hl=en#!topic/zombie-js/cJklyMbwxRE and https://groups.google.com/forum/?hl=en#!topic/zombie-js/tOhk_lZv5eA)
While the latter isn't as important as I can find ways around it (I hope), the former is the main issue, so I'd greatly appreciate any direction on how to resolve this address conflict.
Here's my NodeJS script:
exec = ( require 'child_process' ).exec
fs = require 'fs'
should = require 'should'
yaml = require 'yaml'
Zombie = require 'zombie'
common = require '../../static/assets/js/common'
url = 'ahmeds.local'
browser = new Zombie()
config = null
consoleCb = 'function consoleSuccess(){console.log("success",arguments)}function consoleFailure(){console.log("failure",arguments)}'
browser.debug = true
browser.silent = false
fs.readFile '../../config.yaml', (error, data) ->
config = yaml.eval data.toString 'ascii'
exec 'cd ../../ && dev_appserver.py -a ' + url + ' .', ->
console.log arguments
# browser.visit config.local.url, ->
browser.visit 'http://' + url + ':8080', ->
browser.evaluate consoleCb
browser.evaluate 'profileImage("facebook",consoleSuccess,consoleFailure)'
console.log browser.window.console.output
I have only limited familiarity with NodeJS, but I just tested running a NodeJS server and App Engine local dev server on the same machine — it works just fine. Without seeing your NodeJS code, I'm guessing you're also trying to run NodeJS on port 8080, and so the App Engine server complains when it's started (8080 is the default, and you noted it's the port you are using).
Try passing --port=8081 (or some other port) to your invocation of dev_appserver.py and it should resolve the conflict.
Nothing in the code you've shown (other than the invocation of dev_appserver) should even be listening on any port (unless zombie implements a "server" for remote debugging or something like that). It looks like the port conflict is coming from somewhere else.
Note that zombie's own Mocha test framework does set up an express server, so if you're using it or code lifted from it, that might be doing it.
What does netstat have to say about who's binding to what port?

Resources