I'm working on a project and have come to a bit of an issue due to a limitation of the backend I'm using for me program.
First of all my question is "can I measure the data size of text through URLLoader?".
I'm making an app that is required to receive and send a fair bit of data, but the back end I'm using has limited to me only being able to send 1024 at a time.
The backend is called 'Scoreoid', it's really good for games and user management and such, but i'm using it in a bit of a different way.
All the a side, my issue is, I'm sending data through an array, and I can easily enough break up the array and send it in multiple transactions... but is there a way I can measure the size of the data?
That way I could determine how much of the array I can send at a time.
Here is the code they provide:
function getGame():void
{
var url:String = "API URL";
var request:URLRequest = new URLRequest(url);
var requestVars:URLVariables = new URLVariables();
request.data = requestVars;
requestVars.api_key = "YOUR API KEY";
requestVars.game_id = "YOUR GAME ID";
requestVars.response ="XML"
requestVars.username ="Players Username"
requestVars.key ="Your Key"
requestVars.value ="Key Value"
request.method = URLRequestMethod.POST;
var urlLoader:URLLoader = new URLLoader();
urlLoader = new URLLoader();
urlLoader.dataFormat = URLLoaderDataFormat.TEXT;
urlLoader.addEventListener(Event.COMPLETE, loaderCompleteHandler);
urlLoader.load(request);
}
function loaderCompleteHandler(event:Event):void
{
trace("responseVars: " + event.target.data);
}
URLLoader class has property bytesTotal. You should be able to determine the size of the data through it.
function loaderCompleteHandler(event:Event):void
{
trace("responseVars: " + event.target.data);
trace("size: " + URLLoader(event.target).bytesTotal);
}
Related
I'm using OKHttpClient in a Kotlin app to post a file to an API that gets processed. While the process is running the API is sending back messages to keep the connection alive until the result has been completed. So I'm receiving the following (this is what is printed out to the console using println())
{"status":"IN_PROGRESS","transcript":null,"error":null}
{"status":"IN_PROGRESS","transcript":null,"error":null}
{"status":"IN_PROGRESS","transcript":null,"error":null}
{"status":"DONE","transcript":"Hello, world.","error":null}
Which I believe is being separated by a new line character, not a comma.
I figured out how to extract the data by doing the following but is there a more technically correct way to transform this? I got it working with this but it seems error-prone to me.
data class Status (status : String?, transcript : String?, error : String?)
val myClient = OkHttpClient ().newBuilder ().build ()
val myBody = MultipartBody.Builder ().build () // plus some stuff
val myRequest = Request.Builder ().url ("localhost:8090").method ("POST", myBody).build ()
val myResponse = myClient.newCall (myRequest).execute ()
val myString = myResponse.body?.string ()
val myJsonString = "[${myString!!.replace ("}", "},")}]".replace (",]", "]")
// Forces the response from "{key:value}{key:value}"
// into a readable json format "[{key:value},{key:value},{key:value}]"
// but hoping there is a more technically sound way of doing this
val myTranscriptions = gson.fromJson (myJsonString, Array<Status>::class.java)
An alternative to your solution would be to use a JsonReader in lenient mode. This allows parsing JSON which does not strictly comply with the specification, such as in your case multiple top level values. It also makes other aspects of parsing lenient, but maybe that is acceptable for your use case.
You could then use a single JsonReader wrapping the response stream, repeatedly call Gson.fromJson and collect the deserialized objects in a list yourself. For example:
val gson = GsonBuilder().setLenient().create()
val myTranscriptions = myResponse.body!!.use {
val jsonReader = JsonReader(it.charStream())
jsonReader.isLenient = true
val transcriptions = mutableListOf<Status>()
while (jsonReader.peek() != JsonToken.END_DOCUMENT) {
transcriptions.add(gson.fromJson(jsonReader, Status::class.java))
}
transcriptions
}
Though, if the server continously provides status updates until processing is done, then maybe it would make more sense to directly process the parsed status instead of collecting them all in a list before processing them.
I'm trying to create two spreadsheets: one tracks student attendance at school, the other tracks their attendance at Track practice. The goal is to write a function, that I can set up as a button, that I can click that will automatically send emails to several people if the student is present at school but is absent for sports without getting excused.
Right now, the whole thing is working pretty well, but I have one issue. I have a column that will read "Good" or "Bad" depending on whether the student meets the above condition. The function turns these into an array. I would like to use the index of the "Bad"'s to find the necessary email addresses, which are stored at the same index point in another array that I make from the spreadsheet. I'm not sure how to save this index point and use it to reference the email addresses. Code below.
function sendEmailsMonday() {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("TrackAttendance");
var dataRange = sheet.getRange("D2:D30");
var data = dataRange.getValues();// Gets array of "Good" and "Bad"
for (i in data) {
if(i = "Bad") {
var place = data.indexOf(i);
var dataRange2 = sheet.getRange("M2:M30");// Gets array of email addresses
var data2 = dataRange2.getValues();
var emailAddress = data2[place];
var message = "This is an automated email informing you that your child/advisee ____ was present at school today, but missed Track without being excused. Feel free to email Mr. # with any questions.";
var subject = "___ missed Track Practice";
MailApp.sendEmail(emailAddress, subject, message);
return;
}
}
}
So, the issue comes in with the index lines. If I get rid of
var place = data.indexOf(i);
and replace
var emailAddress = data[place];
with
var emailAddress = data[28];
or any other number, it will grab the email address and send it. But then it has nothing to do with the values in the other column.
Seems like this should be an easy fix but I'm bad at this.
Very late responding now. I think you are almost there.
Your IF statement should read:
if(i == "Bad") {
And then replace 'place' with i:
var emailAddress = data2[i];
It should work as expected now.
I'm working on a project where i need to post the data i acquire to a Google form and obtain the data from the spreadsheet. I cannot use google apps script and need a method using the direct POST method as i will be doing this function from a GSM module. All the solutions posted previously take into consideration the old structure of the Google form which provides a form key.Like the solution described in this one:
http://www.open-electronics.org/how-send-data-from-arduino-to-google-docs-spreadsheet/
The link to my current form is this.
https://docs.google.com/forms/d/14MkYG3fPNezzUC_nXUsWHlZ5JhplvjyWTAeob7f_W7g/viewform
Any help would be appreciated.
Is it a requirement that a google form be in the middle of this? If it is enough to be able to post your data to a spreadsheet, here's a Google-Apps-Script for one side of the problem: a simple web service that will accept form data as a query string, and write that to your spreadsheet.
This examples assumes a very simple spreadsheet, with three columns, "Timestamp", "col1" and "col2". Edit the code to suit your situation.
You can see the spreadsheet here, and even make a test post.
/**
* doGet() function to add data to a spreadsheet.
*
* Spreadsheet data is provided as a querystring, e.g. ?col1=1&col2='pizza'
*
* From: http://stackoverflow.com/a/18725479/1677912
*
* #param {event} e Event passed to doGet, with querystring
* #returns {String/html} Html to be served
*
* Test URLs (adjust ID as needed):
* https://script.google.com/macros/s/--DEV-SCRIPT-ID--/dev?col1=1&col2='pizza'
* https://script.google.com/macros/s/--PUB-SCRIPT-ID--/exec?col1=1&col2='pizza'
*/
function doGet(e) {
Logger.log( JSON.stringify(e) ); // view parameters
var result = 'Ok'; // assume success
if (e.parameter == undefined) {
result = 'No Parameters';
}
else {
var id = '--SHEET-ID---'; // Spreadsheet id for responses
var sheet = SpreadsheetApp.openById(id).getActiveSheet();
var newRow = sheet.getLastRow() + 1;
var rowData = [];
rowData[0] = new Date(); // Timestamp
for (var param in e.parameter) {
Logger.log('In for loop, param='+param);
var value = stripQuotes(e.parameter[param]);
//Logger.log(param + ':' + e.parameter[param]);
switch (param) {
case 'col1':
rowData[1] = value;
break;
case 'col2':
rowData[2] = value;
break;
default:
result = "unsupported parameter";
}
}
Logger.log(JSON.stringify(rowData));
// Write new row to spreadsheet
var newRange = sheet.getRange(newRow, 1, 1, rowData.length);
newRange.setValues([rowData]);
}
// Return result of operation
return ContentService.createTextOutput(result);
}
/**
* Remove leading and trailing single or double quotes
*/
function stripQuotes( value ) {
return value.replace(/^["']|['"]$/g, "");
}
You can do the sending with the new forms, there is a menu option for it. (Responses->Get prefill url) It gives the url for posting data to a form.
You also asked: "obtain the data from the spreadsheet":There are two ways, google apps script and gdata style "google-spreadsheet-api". But I suggest you use a mix of google apps script and "arduino" style code, as it has better docs and features than gdata style api.
p.s. I created some formulas for creating an "arduino" user interface a while back.
First off, thank you #Moishe for the very useful API. I'm having a little timeout problem, maybe someone knows the answer. Here's how I open the channel:
var openChannel = function () {
var channel = new goog.appengine.Channel($('#token').val());
var socket = channel.open();
socket.onopen = function () {};
socket.onmessage = function (m) {
var message = JSON.parse(m.data);
// do stuff
};
socket.onerror = function (error) { alert(error); };
socket.onclose = openChannel;
};
openChannel();
This works fine, I post my messages and they go to the other clients pretty quickly. But if I stay on the page for roughly 15 minutes, the server loses track of my channel. In dev, it throws an error (which I saw was a known bug: http://www.mail-archive.com/google-appengine#googlegroups.com/msg44609.html). But in prod, it still ignores messages on that channel after about 15 minutes.
We fixed it by adding a setInterval(getSomeUrl, everyMinute) to the page, but we'd rather not have to do that. I noticed in Moishe's last commit for the trivia game sample that he took out a keep-alive. I didn't understand how he replaced it, and what he meant by onopen is reliable:
http://code.google.com/p/trivia-quiz/source/browse/trunk/src/index.html
Update: Server side code is
class Home(BaseHandler):
def get(self):
self.checkUser()
if self.user:
userId = self.user.user_id()
token = channel.create_channel(userId)
chatClients[userId] = token
self.model['token'] = token
players = self.checkChatRoom()
self.model['users'] = players
self.model['messages'] = map(lambda k:db.get(k), self.chat_room.messages) # TODO: Replace this line and the next with a query
self.model['messages'] = sorted(self.model['messages'], key=lambda m: m.timestamp, reverse=True)
self.writeTemplate('index.html')
BaseHandler is just a base class I use for all my GAE handlers, it provides checkUser which redirects if the user isn't logged in, and it provides writeTemplate which takes what's in self.model and writes it in a template. It's just a proof of concept, so no cache or anything else other than what's above.
I am new to SalesForce (3 months).
Thus far I have been able to create an application in C# that I can use to preform Inserts and Updates to the SalesForce database. These transactions are one at a time.
No I have the need to preform large scale transactions. For example updating thousands of records at a time. Doing them one by one would quickly put us over our allotted API calls per 24 hour period.
I want to utilize the available bulk transactions process to cut down on the number of API calls. Thus far I have not had much luck coding this nor have I found any such documentation.
If anyone could either provide some generic examples or steer me to reliable documentation on the subject I would greatly appreciate it.
FYI, the data I need to use to do the updates and inserts comes from an IBM Unidata database sitting on an AIX machine. So direct web services communication is not realy possible. Getting the data from Unidata has been my headache. I have that worked out. Now the bulk api to SalesForce is my new headache.
Thanks in advance.
Jeff
You don't mention which API you're currently using, but using the soap partner or enterprise APIs you can write records to salesforce 200 at a time. (the create/update/upsert calls all take an array of SObjects).
Using the bulk API you can send data in chunks of thousands of rows at a time.
You can find the documentation for both sets of APIs here
The answers already given are a good start; however, are you sure you need to actually write a custom app that uses the bulk API? The salesforce data loader is a pretty robust tool, includes a command line interface, and can use either the "normal" or bulk data API's. Unless you are needing to do fancy logic as part of your insert/updates, or some sort of more real-time / on-demand loading, the data loader is going to be a better option than a custom app.
(this is the SOAP code though, not the Salesforce "Bulk API" ; careful not to confuse the two)
Mighy be below code provide clear insight on how to do bulk insertion.
/// Demonstrates how to create one or more Account records via the API
public void CreateAccountSample()
{
Account account1 = new Account();
Account account2 = new Account();
// Set some fields on the account1 object. Name field is not set
// so this record should fail as it is a required field.
account1.BillingCity = "Wichita";
account1.BillingCountry = "US";
account1.BillingState = "KA";
account1.BillingStreet = "4322 Haystack Boulevard";
account1.BillingPostalCode = "87901";
// Set some fields on the account2 object
account2.Name = "Golden Straw";
account2.BillingCity = "Oakland";
account2.BillingCountry = "US";
account2.BillingState = "CA";
account2.BillingStreet = "666 Raiders Boulevard";
account2.BillingPostalCode = "97502";
// Create an array of SObjects to hold the accounts
sObject[] accounts = new sObject[2];
// Add the accounts to the SObject array
accounts[0] = account1;
accounts[1] = account2;
// Invoke the create() call
try
{
SaveResult[] saveResults = binding.create(accounts);
// Handle the results
for (int i = 0; i < saveResults.Length; i++)
{
// Determine whether create() succeeded or had errors
if (saveResults[i].success)
{
// No errors, so retrieve the Id created for this record
Console.WriteLine("An Account was created with Id: {0}",
saveResults[i].id);
}
else
{
Console.WriteLine("Item {0} had an error updating", i);
// Handle the errors
foreach (Error error in saveResults[i].errors)
{
Console.WriteLine("Error code is: {0}",
error.statusCode.ToString());
Console.WriteLine("Error message: {0}", error.message);
}
}
}
}
catch (SoapException e)
{
Console.WriteLine(e.Code);
Console.WriteLine(e.Message);
}
}
Please find the small code which may help you to insert the data into salesforce objects using c# and WSDL APIs. I stuck to much to write code in c#. I assigned using direct index after spiting you can use your ways.
I split the column using | (pipe sign). You may change this and also <br>, \n, etc. (row and column breaking)
Means you can enter N rows which are in your HTML/text file. I wrote the program to add order by my designers who put the order on other website and fetch the data from e-commerce website and who has no interface for the salesforce to add/view the order records. I created one object for the same. and add following columns in the object.
Your suggestions are welcome.
private SforceService binding; // declare the salesforce servive using your access credential
try
{
string stroppid = "111111111111111111";
System.Net.HttpWebRequest fr;
Uri targetUri = new Uri("http://abc.xyz.com/test.html");
fr = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(targetUri);
if ((fr.GetResponse().ContentLength > 0))
{
System.IO.StreamReader str = new System.IO.StreamReader(fr.GetResponse().GetResponseStream());
string allrow = str.ReadToEnd();
string stringSeparators = "<br>";
string[] row1 = Regex.Split(allrow, stringSeparators);
CDI_Order_Data__c[] cord = new CDI_Order_Data__c[row1.Length - 1];
for (int i = 1; i < row1.Length-1; i++)
{
string colstr = row1[i].ToString();
string[] allcols = Regex.Split(colstr, "\\|");
cord[i] = new CDI_Order_Data__c(); // Very important to create object
cord[i].Opportunity_Job_Order__c = stroppid;
cord[i].jobid__c = stroppid;
cord[i].order__c = allcols[0].ToString();
cord[i].firstname__c = allcols[1].ToString();
cord[i].name__c = allcols[2].ToString();
DateTime dtDate = Convert.ToDateTime(allcols[3]);
cord[i].Date__c = new DateTime(Convert.ToInt32(dtDate.Year), Convert.ToInt32(dtDate.Month), Convert.ToInt32(dtDate.Day), 0, 0, 0); //sforcedate(allcols[3]); //XMLstringToDate(allcols[3]);
cord[i].clientpo__c = allcols[4].ToString();
cord[i].billaddr1__c = allcols[5].ToString();
cord[i].billaddr2__c = allcols[6].ToString();
cord[i].billcity__c = allcols[7].ToString();
cord[i].billstate__c = allcols[8].ToString();
cord[i].billzip__c = allcols[9].ToString();
cord[i].phone__c = allcols[10].ToString();
cord[i].fax__c = allcols[11].ToString();
cord[i].email__c = allcols[12].ToString();
cord[i].contact__c = allcols[13].ToString();
cord[i].lastname__c = allcols[15].ToString();
cord[i].Rep__c = allcols[16].ToString();
cord[i].sidemark__c = allcols[17].ToString();
cord[i].account__c = allcols[18].ToString();
cord[i].item__c = allcols[19].ToString();
cord[i].kmatid__c = allcols[20].ToString();
cord[i].qty__c = Convert.ToDouble(allcols[21]);
cord[i].Description__c = allcols[22].ToString();
cord[i].price__c = Convert.ToDouble(allcols[23]);
cord[i].installation__c = allcols[24].ToString();
cord[i].freight__c = allcols[25].ToString();
cord[i].discount__c = Convert.ToDouble(allcols[26]);
cord[i].salestax__c = Convert.ToDouble(allcols[27]);
cord[i].taxcode__c = allcols[28].ToString();
}
try {
SaveResult[] saveResults = binding.create(cord);
}
catch (Exception ce)
{
Response.Write("Buld order update errror" +ce.Message.ToString());
Response.End();
}
if (str != null) str.Close();
}