Trying to Export Datatable to Excel (1.000.000 Records) in Web API and Angular, Writing Streaming is throwing an Out of Memory Exception - angularjs

Trying to Export Datatable to Excel (1.000.000 Records) in Web API and Angular, Writing Streaming is throwing an Out of Memory Exception.
Tried Splitting Datatables with 100.000 records each and writing stream seperately. However when I try to concat the stream, Out Of Memory Exception occurs.
Is it ever possible to export 1.000.000 records in an excel sheet. I am able to export upto 175.000 records. But require 1.000.000. See Snippet Below.
[HttpPost]
public HttpResponseMessage DownloadExcel([FromBody]AuditRequest auditRequest)
{
HttpResponseMessage result = Request.CreateResponse(HttpStatusCode.OK);
var response = GetAuditForDownload(auditRequest);
MemoryStream streamFinal = new MemoryStream();
List<AuditRequest> auditListForBulk = new List<AuditRequest>();
foreach (AuditRequest req in response.Data)
{
req.FromDate = auditRequest.FromDate;
req.ToDate = auditRequest.ToDate;
}
DataTable table = ConvertToDataTable(response.Data);
List<DataTable> splittedtables = table.AsEnumerable()
.Select((row, index) => new { row, index })
.GroupBy(x => x.index / 100000) //integer division, the fractional part is truncated
.Select(g => g.Select(x => x.row).CopyToDataTable())
.ToList();
splittedtables.ForEach(delegate (DataTable SplitTable)
{
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(ToCSV(SplitTable).ToString());
writer.Flush();
stream.Position = 0;
stream.CopyTo(streamFinal);
//byteArray = GetBytesFromDataSet(SplitTable);
//byteArrayFinal = byteArrayFinal.Concat(byteArray).ToArray();
});
result = new HttpResponseMessage(HttpStatusCode.OK) { Content = new ByteArrayContent(streamFinal.ToArray()) };
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = "File.csv"
};
return result;
}

Related

How to read spatial data types from SqlCommand?

In my .NET Core application, I'm trying to read spatial data types (NetTopologySuite's Geometry) from a SqlCommand object using this code:
using (var result = command.ExecuteReader())
{
var entities = new List<Geometry>();
while (result.Read())
{
var geometryReader = new SqlServerBytesReader { IsGeography = false };
var bytes = result.GetSqlBytes(0).Value;
var geometry = geometryReader.Read(bytes);
entities.Add(geometry);
}
connection.Close();
return entities;
}
It's working fine. But the problem that I can't use a single generic method for all not spatial data types:
using (var result = command.ExecuteReader())
{
var entities = new List<T>();
while (result.Read())
{
entities.Add((T)result.GetValue(0));
}
connection.Close();
return entities;
}
I tried to use this snippet to read Geometry but got an error:
Unable to cast object of type 'Microsoft.SqlServer.Types.SqlGeometry' to type 'Microsoft.Data.SqlClient.Server.IBinarySerialize'.
Am I missing something and is it possible to read all spatial data types with the same generic code?

Web API 2, return string as a file

I have a Web API 2 POST endpoint which takes a parameter, queries the database and returns an xml string as the response.
public async Task<IHttpActionResult> Post(long groupId)
{
People people = await _someService.GetPeople(groupId);
XElement peopleXml = _xmlService.ConverToXml(people);
return Ok(peopleXml);
}
How do I to return the xml as a file instead?
Figured it out myself, but I hope there is a simpler way -
public async Task<IHttpActionResult> Post(long groupId)
{
People people = await _someService.GetPeople(groupId);
XElement peopleXml = _xmlService.ConverToXml(people);
byte[] toBytes = Encoding.Unicode.GetBytes(peopleXml.ToString());
var stream = new MemoryStream(toBytes);
var result = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StreamContent(stream)
};
result.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment")
{
FileName = "test.txt"
};
result.Content.Headers.ContentType = new MediaTypeHeaderValue("text/plain");
var response = ResponseMessage(result);
return response;
}

How to display images in web page using angularjs?

I already know how to save images in mongodb using angularjs and java to save it in my mongodb,
I need to get the saved image from mongodb and display it in an html page using AngularJS.
This is my controller for getting image
#GET
#Path("/{id}")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getById(#PathParam("id") String id) throws IOException
{
Response response = null;
MongoClient mongoClient = new MongoClient("localhost", 27017);
DB mongoDB = mongoClient.getDB("sampleDB");
DBCollection collection = mongoDB.getCollection("filestore");
BasicDBObject query = new BasicDBObject();
ObjectId oid = new ObjectId(id);
query.put("_id", oid);
GridFS fileStore = new GridFS(mongoDB, "filestore");
GridFSDBFile gridFile = fileStore.findOne(query);
InputStream in = gridFile.getInputStream();
ByteArrayOutputStream out = new ByteArrayOutputStream();
int data = in.read();
while (data >= 0)
{
out.write((char) data);
data = in.read();
}
out.flush();
ResponseBuilder builder = Response.ok(out.toByteArray());
builder.header("Content-Disposition", "attachment; filename=");
response = builder.build();
return response;
}
This is my angularjs for getting image
var userImagePromise = $http.get("../api/s3/" + $scope.user.profileImage[0].id);
userImagePromise.success(function(data, status, headers, config) {
$scope.imageData = data;
});
userImagePromise.error(function(data, status, headers, config) {
});
This is my html for displaying image
<img id="userProfileImg" height="150px" width="150px" ng-src="data:image/png;base64,{{imageData}}">
if I simply put the link to browser i got this output in octect-stream
�PNG .....
How to display image in html?Any error in my code wise?
Image for getting output
i think your base64 code is not converting images properly, so check my code it may help you.
import java.awt.image.BufferedImage;
import java.io.BufferedWriter;
import java.io.ByteArrayOutputStream;
import javax.imageio.ImageIO;
BufferedImage buffimage = ImageIO.read(new File(imagePath));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(buffimage, "png", baos);
String img = Base64.encode(baos.toByteArray());
send this img variable to your angularjs code.

How to save file in windows 8

Hello I'm learning windows phone 8 and i have problem with saving file, basicaly I'am downloading zip file from web service and i need to save it, but i was not able to do that. I want to save contents of inStream Any suggestions ?
private async Task saveSetupFile(Stream inStream)
{
StorageFolder local = Windows.Storage.ApplicationData.Current.LocalFolder;
var myFolder = await local.CreateFolderAsync("AuditsSetupFiles", CreationCollisionOption.OpenIfExists);
var file = await myFolder.CreateFileAsync("EquipSetup.zip", CreationCollisionOption.ReplaceExisting);
var outStream = await file.OpenStreamForWriteAsync();
byte [] buffer = new byte[1024];
int len;
using (var s = await file.OpenStreamForWriteAsync())
{
}
This should work:
private async Task SaveSetupFile(Stream inStream)
{
var localFolder = ApplicationData.Current.LocalFolder;
var folder = await localFolder.CreateFolderAsync("AuditsSetupFiles", CreationCollisionOption.OpenIfExists);
var file = await folder.CreateFileAsync("EquipSetup.zip", CreationCollisionOption.ReplaceExisting);
using (var outStream = await file.OpenStreamForWriteAsync())
{
await inStream.CopyToAsync(outStream);
}
}

Facebook Photo Upload in C#?

I am trying to upload an photo to Facebook from a Windows Phone Silverlight application using the Facebook Graph API but I am getting an error: (#324) Requires upload file. Can anyone see anything wrong in my code?
internal void PublishPhoto(System.IO.MemoryStream stream, string message, string accessToken)
{
var requestUriString = string.Format(
CultureInfo.InvariantCulture,
"https://graph.facebook.com/{0}/photos?access_token={1}&message={2}",
"me",
accessToken,
message);
var webRequest = WebRequest.CreateHttp(requestUriString);
webRequest.Method = "POST";
var boundary = "7db3d9202a1";
webRequest.ContentType = string.Format("multipart/form-data; boundary={0}", boundary);
webRequest.BeginGetRequestStream(new AsyncCallback(delegate (IAsyncResult result)
{
GetRequestStream(stream, boundary, result);
BeginGetResponse(webRequest);
}), webRequest);
}
private static void GetRequestStream(System.IO.MemoryStream imageStream, string boundary, IAsyncResult result)
{
var webRequest2 = result.AsyncState as HttpWebRequest;
using (var requestStream = webRequest2.EndGetRequestStream(result))
{
using (StreamWriter writer = new StreamWriter(requestStream))
{
writer.WriteLine("--{0}\r", boundary);
writer.WriteLine("Content-Disposition: form-data; filename=\"sketch.jpg\"\r");
writer.WriteLine("Content-Type: image/jpg\r");
byte[] buffer = imageStream.GetBuffer();
requestStream.Write(buffer, 0, buffer.Length);
writer.WriteLine("\r");
writer.WriteLine("--{0}--\r", boundary);
}
imageStream.Close();
}
}
private static void BeginGetResponse(HttpWebRequest webRequest)
{
webRequest.BeginGetResponse(new AsyncCallback(delegate(IAsyncResult result2)
{
var webRequest2 = result2.AsyncState as HttpWebRequest;
try
{
using (var response = webRequest2.EndGetResponse(result2))
{
using (var responseStream = response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(responseStream))
{
System.Diagnostics.Debug.WriteLine(reader.ReadToEnd());
}
}
}
}
catch (WebException we)
{
System.Diagnostics.Debug.WriteLine(we.Message);
using (var responseStream = we.Response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(responseStream))
{
var errorJson = reader.ReadToEnd();
var response = Newtonsoft.Json.JsonConvert.DeserializeObject<FacebookErrorResponse>(errorJson);
System.Diagnostics.Debug.WriteLine("Could not upload image to Facebook: {0}", response.Error.Message);
}
}
}
}), webRequest);
}
}
Try specifying a name of "source" as well as a filename in the Content-Disposition header, i.e.
writer.WriteLine("Content-Disposition: form-data; name=\"source\"; filename=\"sketch.jpg\"\r");
Ok, I was wrong the first time around, but now I have it.
The first problem, which we already took care of above, was that you were missing the "--" before each boundary and the "--" after the last boundary in the POST body.
The second problem is that you're not leaving a blank line after the MIME headers before writing the image content.
The third problem is that you're not flushing the writer before writing the image data to its underlying stream (unless silverlight on a phone is different from normal .NET in auto-flushing StreamWriters).
To sum up, this should work:
writer.WriteLine("--{0}\r", boundary);
writer.WriteLine("Content-Disposition: form-data; filename=\"sketch.jpg\"\r");
writer.WriteLine("Content-Type: image/jpg\r");
writer.WriteLine("\r");
writer.Flush();
byte[] buffer = imageStream.GetBuffer();
requestStream.Write(buffer, 0, buffer.Length);
writer.WriteLine("\r");
writer.WriteLine("--{0}--\r", boundary);

Resources