Is it possible to perform oneHotDecoder after using OneHotEncoder in spark ml? Is there any way to achieve this?
StringIndexer dateIndexer = new StringIndexer();
csvData = dateIndexer.setInputCol("Date").setOutputCol("dateIndex").fit(csvData).transform(csvData);
StringIndexer timeIndexer = new StringIndexer();
csvData = timeIndexer.setInputCol("Time").setOutputCol("timeIndex").fit(csvData).transform(csvData);
OneHotEncoderEstimator encoder = new OneHotEncoderEstimator(); csvData = encoder.setInputCols(new String[] { "dateIndex", "timeIndex"}).setOutputCols(new String[] { "dateVector", "timeVector"}).fit(csvData).transform(csvData);
I could not find any solution on this in the spark API docs.
Related
I have a string containing a list of thousands of geo coordinates
"[8.2627925,46.3460497],[8.2612673,46.3457009],[8.2602494,46.3453934],..."
and would like to extract all of them in an array of CGPoints.
The code I am using today is the following:
let coords=[CGPoint]()
while substring.contains("[")
{
index = substring.index(of: "[")!
substring = substring[index...]
index = substring.index(of: "]")!
let coordinate = substring[..<index]
var indexComma = coordinate.index(of: ",")!
let coordX = String(coordinate[..<indexComma])
indexComma = coordinate.index(indexComma, offsetBy: +1)
let coordY = String(coordinate[indexComma...])
coords.append(CGPoint(x: CGFloat(Double(coordX)!), y: CGFloat(Double(coordY)!)))
substring = substring[index...]
}
However I am sure it can be optimised, possibly using regex. What would be the most efficient way ?
One of the better efficient way is using to json serialization. Json can not serialize your string because its not in to correct format so you can insert "[" and "]" for serialize it like below
var coordinates = "[1.3123,3.2131],[2.3123,4.213]"
coordinates.insert("[", at: coordinates.startIndex)
coordinates.insert("]", at: coordinates.endIndex)
Now , our data is ready for serialization.Then the easiest part is appending poinst to CGPoint array
var points : [CGPoint] = []
if let data = coordinates.data(using: .utf8),
let jsonArray = try? JSONSerialization.jsonObject(with: data) as? [[Double]] {
points.append(contentsOf: jsonArray.map{CGPoint(x: $0[0], y: $0[1]) })
print(points)
}
You can easily reach ur datas
print(points[0].x) // 1.3123
print(points[1].y) // 4.213
Using shared preferences to save an arrayListOf in my app. Im converting the array to a hashset and saving it but when I load it I am unable to convert it back to an arrayListOf. I have tried toTypedArray() and toArray() but neither solve the problem. Getting the error below.
Required:
kotlin.collections.ArrayList /* = java.util.ArrayList */
Found:
(Mutable)Set<String!>?
Using the following to try and load the hashet and convert it.
var sharedPreferences = getSharedPreferences(SHARED_PREFS, MODE_PRIVATE)
var places = arrayListOf<String>()
var loadSet = sharedPreferences.getStringSet("key", null)
places = loadSet
go.setOnClickListener{
thread {
val place = newLocation.text.toString()
var stringCounter = counter.toString()
if(place !in places){
places.add(place)
//editor.putStringSet("key", places)
//editor.commit()
var sharedPreferences = getSharedPreferences(SHARED_PREFS, MODE_PRIVATE)
var editor = sharedPreferences.edit()
set.add(place)
editor.putStringSet("key", set);
editor.commit();
Log.d("set", set.toString())
var loadSet = sharedPreferences.getStringSet("key", HashSet<String>())?.toList()
Log.d("load", loadSet.toString())
places = loadSet as ArrayList<String>
}
arrayListOf isn’t a type. It’s a function. The type is ArrayList, but it’s more typical to only need the more abstract List or MutableList, depending on what you’re doing with it. For those you can use set.toList() or set.toMutableList().
If you have an existing ArrayList or MutableList, you could also do list.addAll(set).
I am attempting to follow this example in order to buffer some features drawn on the map.
However, that gives me the following error:
ERROR TypeError: Cannot read property 'Point' of undefined
Here is my code:
BufferFeature(feature){
var parser = new jsts.io.OL3Parser();
let format = new ol.format.WKT();
// convert the OpenLayers geometry to a JSTS geometry
var jstsGeom = parser.read(format.writeGeometry(feature.getGeometry()));
// create a buffer of 40 meters around each line
var buffered = jstsGeom.buffer(40);
// convert back from JSTS and replace the geometry on the feature
feature.setGeometry(parser.write(buffered));
let source = this.VectorOverlay.getSource();
source.addFeatures(feature);
}
Can someone identify where I am going wrong?
I finally solve this problem using the inject method of OL3Parse. your code should look like:
var Geojson = require('ol/format/geojson');
var Vector = require('ol/source/vector');
var Feature = require('ol/feature');
var LineString = require('ol/geom/linestring');
var Point = require('ol/geom/point');
var jsts = require('jsts');
var ol3Parser = new jsts.io.OL3Parser();
ol3Parser.inject(Point, LineString);
BufferFeature(feature){
let format = new ol.format.WKT();
// convert the OpenLayers geometry to a JSTS geometry
var jstsGeom = ol3Parser.read(format.writeGeometry(feature.getGeometry()));
// create a buffer of 40 meters around each line
var buffered = jstsGeom.buffer(40);
// convert back from JSTS and replace the geometry on the feature
feature.setGeometry(parser.write(buffered));
let source = this.VectorOverlay.getSource();
source.addFeatures(feature);
}
This works for:
function inject(Point, LineString, LinearRing, Polygon, MultiPoint, MultiLineString, MultiPolygon, GeometryCollection) {};
I am new to SalesForce (3 months).
Thus far I have been able to create an application in C# that I can use to preform Inserts and Updates to the SalesForce database. These transactions are one at a time.
No I have the need to preform large scale transactions. For example updating thousands of records at a time. Doing them one by one would quickly put us over our allotted API calls per 24 hour period.
I want to utilize the available bulk transactions process to cut down on the number of API calls. Thus far I have not had much luck coding this nor have I found any such documentation.
If anyone could either provide some generic examples or steer me to reliable documentation on the subject I would greatly appreciate it.
FYI, the data I need to use to do the updates and inserts comes from an IBM Unidata database sitting on an AIX machine. So direct web services communication is not realy possible. Getting the data from Unidata has been my headache. I have that worked out. Now the bulk api to SalesForce is my new headache.
Thanks in advance.
Jeff
You don't mention which API you're currently using, but using the soap partner or enterprise APIs you can write records to salesforce 200 at a time. (the create/update/upsert calls all take an array of SObjects).
Using the bulk API you can send data in chunks of thousands of rows at a time.
You can find the documentation for both sets of APIs here
The answers already given are a good start; however, are you sure you need to actually write a custom app that uses the bulk API? The salesforce data loader is a pretty robust tool, includes a command line interface, and can use either the "normal" or bulk data API's. Unless you are needing to do fancy logic as part of your insert/updates, or some sort of more real-time / on-demand loading, the data loader is going to be a better option than a custom app.
(this is the SOAP code though, not the Salesforce "Bulk API" ; careful not to confuse the two)
Mighy be below code provide clear insight on how to do bulk insertion.
/// Demonstrates how to create one or more Account records via the API
public void CreateAccountSample()
{
Account account1 = new Account();
Account account2 = new Account();
// Set some fields on the account1 object. Name field is not set
// so this record should fail as it is a required field.
account1.BillingCity = "Wichita";
account1.BillingCountry = "US";
account1.BillingState = "KA";
account1.BillingStreet = "4322 Haystack Boulevard";
account1.BillingPostalCode = "87901";
// Set some fields on the account2 object
account2.Name = "Golden Straw";
account2.BillingCity = "Oakland";
account2.BillingCountry = "US";
account2.BillingState = "CA";
account2.BillingStreet = "666 Raiders Boulevard";
account2.BillingPostalCode = "97502";
// Create an array of SObjects to hold the accounts
sObject[] accounts = new sObject[2];
// Add the accounts to the SObject array
accounts[0] = account1;
accounts[1] = account2;
// Invoke the create() call
try
{
SaveResult[] saveResults = binding.create(accounts);
// Handle the results
for (int i = 0; i < saveResults.Length; i++)
{
// Determine whether create() succeeded or had errors
if (saveResults[i].success)
{
// No errors, so retrieve the Id created for this record
Console.WriteLine("An Account was created with Id: {0}",
saveResults[i].id);
}
else
{
Console.WriteLine("Item {0} had an error updating", i);
// Handle the errors
foreach (Error error in saveResults[i].errors)
{
Console.WriteLine("Error code is: {0}",
error.statusCode.ToString());
Console.WriteLine("Error message: {0}", error.message);
}
}
}
}
catch (SoapException e)
{
Console.WriteLine(e.Code);
Console.WriteLine(e.Message);
}
}
Please find the small code which may help you to insert the data into salesforce objects using c# and WSDL APIs. I stuck to much to write code in c#. I assigned using direct index after spiting you can use your ways.
I split the column using | (pipe sign). You may change this and also <br>, \n, etc. (row and column breaking)
Means you can enter N rows which are in your HTML/text file. I wrote the program to add order by my designers who put the order on other website and fetch the data from e-commerce website and who has no interface for the salesforce to add/view the order records. I created one object for the same. and add following columns in the object.
Your suggestions are welcome.
private SforceService binding; // declare the salesforce servive using your access credential
try
{
string stroppid = "111111111111111111";
System.Net.HttpWebRequest fr;
Uri targetUri = new Uri("http://abc.xyz.com/test.html");
fr = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(targetUri);
if ((fr.GetResponse().ContentLength > 0))
{
System.IO.StreamReader str = new System.IO.StreamReader(fr.GetResponse().GetResponseStream());
string allrow = str.ReadToEnd();
string stringSeparators = "<br>";
string[] row1 = Regex.Split(allrow, stringSeparators);
CDI_Order_Data__c[] cord = new CDI_Order_Data__c[row1.Length - 1];
for (int i = 1; i < row1.Length-1; i++)
{
string colstr = row1[i].ToString();
string[] allcols = Regex.Split(colstr, "\\|");
cord[i] = new CDI_Order_Data__c(); // Very important to create object
cord[i].Opportunity_Job_Order__c = stroppid;
cord[i].jobid__c = stroppid;
cord[i].order__c = allcols[0].ToString();
cord[i].firstname__c = allcols[1].ToString();
cord[i].name__c = allcols[2].ToString();
DateTime dtDate = Convert.ToDateTime(allcols[3]);
cord[i].Date__c = new DateTime(Convert.ToInt32(dtDate.Year), Convert.ToInt32(dtDate.Month), Convert.ToInt32(dtDate.Day), 0, 0, 0); //sforcedate(allcols[3]); //XMLstringToDate(allcols[3]);
cord[i].clientpo__c = allcols[4].ToString();
cord[i].billaddr1__c = allcols[5].ToString();
cord[i].billaddr2__c = allcols[6].ToString();
cord[i].billcity__c = allcols[7].ToString();
cord[i].billstate__c = allcols[8].ToString();
cord[i].billzip__c = allcols[9].ToString();
cord[i].phone__c = allcols[10].ToString();
cord[i].fax__c = allcols[11].ToString();
cord[i].email__c = allcols[12].ToString();
cord[i].contact__c = allcols[13].ToString();
cord[i].lastname__c = allcols[15].ToString();
cord[i].Rep__c = allcols[16].ToString();
cord[i].sidemark__c = allcols[17].ToString();
cord[i].account__c = allcols[18].ToString();
cord[i].item__c = allcols[19].ToString();
cord[i].kmatid__c = allcols[20].ToString();
cord[i].qty__c = Convert.ToDouble(allcols[21]);
cord[i].Description__c = allcols[22].ToString();
cord[i].price__c = Convert.ToDouble(allcols[23]);
cord[i].installation__c = allcols[24].ToString();
cord[i].freight__c = allcols[25].ToString();
cord[i].discount__c = Convert.ToDouble(allcols[26]);
cord[i].salestax__c = Convert.ToDouble(allcols[27]);
cord[i].taxcode__c = allcols[28].ToString();
}
try {
SaveResult[] saveResults = binding.create(cord);
}
catch (Exception ce)
{
Response.Write("Buld order update errror" +ce.Message.ToString());
Response.End();
}
if (str != null) str.Close();
}
All,
I wrote a method that extends some of the functionality in the SolrNet example MVC app. It's very simple and in my controller I added the following.
public string Index()
{
try
{
var solr = ServiceLocator.Current.GetInstance>();
String qs = Request.QueryString["q"];
var results = solr.Query(new SolrQueryByField("asciiname", qs));
StringBuilder buffer = new StringBuilder();
buffer.Append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
buffer.Append("<kml xmlns=\"http://www.opengis.net/kml/2.2\"\n");
buffer.Append(" xmlns:gx=\"http://www.google.com/kml/ext/2.2\">\n");
buffer.Append("<Document>\n");
buffer.Append("<name>\n");
buffer.Append("Gazetteer\n");
buffer.Append("</name>\n");
buffer.Append("<description>\n");
buffer.Append("World Places\n");
buffer.Append("</description>\n");
foreach (var i in results)
{
String lat = i.latitude.ToString();
String lon = i.longitude.ToString();
String name = i.asciiName.ToString();
String coords = lon.ToString() + "," + lat.ToString();
name = name.Replace("&", "&");
name = name.Replace("<", "<");
name = name.Replace(">", ">");
buffer.Append("<Placemark>\n");
buffer.Append("<name>\n");
buffer.Append(name);
buffer.Append("</name>\n");
buffer.Append("<Point>\n");
buffer.Append("<coordinates>\n");
buffer.Append(coords);
buffer.Append("</coordinates>\n");
buffer.Append("</Point>\n");
buffer.Append("</Placemark>\n");
}
buffer.Append("</Document>\n");
buffer.Append("</kml>");
Response.Write(buffer.ToString());
Response.AddHeader("content-disposition", "attachment; filename=geonames.kml");
Response.ContentType = "application/vnd.google-earth.kml+xml";
Response.AppendHeader("Content-Encoding", "kml");
return buffer.ToString();
}
catch (Exception)
{
return "ERROR";
}
}
The KML file is generated but the content is duplicated starting at the tag. Is this happening in var solr = ServiceLocator.Current.GetInstance>(); for some reason? I have been walking through the debugger all day and can't seem to figure out why it's writing out the whole thing twice. It does utilize the foreach loop and iterate properly through that but again, it's duplicating the contents of the entire file.
Any help on this would be great!
Adam
Ughhh...I completely overlooked the fact that I was writing the buffer out twice!
Response.Write(buffer.ToString());
and
return buffer.ToString();
Adam