I am creating an external stage, I want it to be based on 2 URLs.
PROBLEM 1
url1 = s3://bucket1/f1/2022/2/
url2 = s3://bucket1/f3/2022/2/
create or replace stage ext_stage url = ??????????
file_format=data_format
storage_integration=s3_integration;
How can I give 2 URLs in the external stage command? Is it possible?
PROBLEM 2
Also, I need to form the URLs.
I am thinking to use a procedure for it.
CREATE OR REPLACE PROCEDURE get_url()
RETURNS STRING
LANGUAGE SCALA
RUNTIME_VERSION = '2.12'
HANDLER = 'C.run'
PACKAGES = ('com.snowflake:snowpark:latest')
AS
$$
import java.util.Calendar
object C{
def run(session: com.snowflake.snowpark.Session): String = {
try {
val cal = Calendar.getInstance()
val year =cal.get(Calendar.YEAR)
var month =cal.get(Calendar.MONTH) + 1
return "s3://bucket1/folder1/" + year + "/"+ month + "/"
}
catch {
case e: Throwable => println("Not able to for url")
return "Failed"
}
}
}
$$;
create or replace stage ext_stage url = call get_url()
file_format=data_format
storage_integration=s3_integration;
It is failing and I can not call the function. How can I call it?
Related
I have a simulation with a step that allows me to publish to different endpoints.
class MySimulation extends Simulation {
// some init code
var testTitle = this.getClass.getSimpleName
val myscenario = scenario("Scn Description")
.exec(PublishMessageRandom(pConfigTest, testTitle + "-" + numProducers, numProducers))
if (testMode == "debug") {
setUp(
myscenario.inject(
atOnceUsers(1)
)
).protocols(httpConf)
} else if (testMode == "open") {
setUp(
myscenario.inject(
rampConcurrentUsers(concurrentUserMin) to (concurrentUserMax) during (durationInMinutes minutes),
)
).protocols(httpConf)
}
}
Now here is my PublishMessageRandom definition
def PublishMessageRandom(producerConfig : ProducerConfig, testTitle : String, numberOfProducers : Int ) = {
val jsonBody = producerConfig.asJson
val valuedJsonBody = Printer.noSpaces.copy(dropNullValues = true).print(jsonBody)
println(valuedJsonBody)
val nodes : Array[String] = endpoints.split(endpointDelimiter)
val rnd = scala.util.Random
val rndIndex = rnd.nextInt(numberOfProducers)
var endpoint = "http://" + nodes(rndIndex) + perfEndpoint
println("endpoint:" + endpoint)
exec(http(testTitle)
.post(endpoint)
.header(HttpHeaderNames.ContentType, HttpHeaderValues.ApplicationJson)
.body(StringBody(valuedJsonBody))
.check(status.is(200))
.check(bodyString.saveAs("serverResponse"))
)
// the below is only useful in debug mode. Comment it out for longer tests
/*.exec { session =>
println("server_response: " + session("serverResponse").as[String])
println("endpoint:" + endpoint)
session */
}
}
as you can see it simply round-robin of endpoints. Unfortunately I see the above println("endpoint:" + endpoint) once and it looks like it picks one endpoint randomly and keeps hitting that instead of desired purpose of hitting endpoints randomly.
Can someone explain that behavior? Is Gatling caching the Step or and how do I go around that?
Quoting the official documentation:
Warning
Gatling DSL components are immutable ActionBuilder(s) that have to be
chained altogether and are only built once on startup. The results is
a workflow chain of Action(s). These builders don’t do anything by
themselves, they don’t trigger any side effect, they are just
definitions. As a result, creating such DSL components at runtime in
functions is completely meaningless.
I had to use feeder to solve the problem where the feeder takes the random endpoint.
// feeder is random endpoint as per number of producers
val endpointFeeder = GetEndpoints(numProducers).random
val myscenario = scenario("Vary number of producers hitting Kafka cluster")
.feed(endpointFeeder)
.exec(PublishMessageRandom(pConfigTest, testTitle + "-" + numProducers))
and Publish message random looks like this:
def PublishMessageRandom(producerConfig : ProducerConfig, testTitle : String ) = {
val jsonBody = producerConfig.asJson
val valuedJsonBody = Printer.noSpaces.copy(dropNullValues = true).print(jsonBody)
println(valuedJsonBody)
exec(http(testTitle)
.post("${endpoint}")
.header(HttpHeaderNames.ContentType, HttpHeaderValues.ApplicationJson)
.body(StringBody(valuedJsonBody))
.check(status.is(200))
.check(bodyString.saveAs("serverResponse"))
)
}
you see the line above .post("${endpoint}") will end up hitting the endpoint coming from the feeder.
The feeder function GetEndpoints is defined as follows
where we create an array of maps with one value each "endpoint" is the key.
def GetEndpoints(numberOfProducers : Int ) : Array[Map[String,String]] = {
val nodes : Array[String] = endpoints.split(endpointDelimiter)
var result : Array[Map[String,String]] = Array()
for( elt <- 1 to numberOfProducers ) {
var endpoint = "http://" + nodes(elt-1) + perfEndpoint
var m : Map[String, String] = Map()
m += ("endpoint" -> endpoint )
result = result :+ m
println("map:" + m)
}
result
}
I want to get values in my code from a static resource file.
I am using an excel file in a static resource, I want to know how I can map fields in an excel file to salesforce fields in my code. for example Account.Name etc
Lead l = new Lead();
l.Company = $Resource.EMailServicesLead.EmailServiceStaticResourceData.csv;
\ Not Working
EMailServicesLead - Name of the static resource.
EmailServiceStaticResourceData - csv file with fields {FirstName, LastName, Email, Company, Status, LeadSource}
/**
* #File Name : myHandler.cls
* #Description :
* #Author : ChangeMeIn#UserSettingsUnder.SFDoc
* #Group :
* #Last Modified By : ChangeMeIn#UserSettingsUnder.SFDoc
* #Last Modified On : 7/10/2019, 3:54:01 PM
* #Modification Log :
* Ver Date Author Modification
* 1.0 7/10/2019 ChangeMeIn#UserSettingsUnder.SFDoc Initial Version
**/
global with sharing virtual class myHandler implements Messaging.InboundEmailHandler
{
global Messaging.InboundEmailResult handleInboundEmail(Messaging.InboundEmail email, Messaging.InboundEnvelope envelope)
{ Messaging.InboundEmailResult result = new Messaging.InboundEmailresult();
//Lead l= (Lead)ListofLeadsStaticRes[0];
// List<SObject> ListofLeadsStaticRes = {!URLFOR($Resource.EMailServicesLead, 'EmailServiceStaticResourceData.csv')};
Integer j=0;
String myPlainText= '';
staticResource sr = new staticResource();
List<sObject> ls = Test.loadData(Lead.sObjectType, 'myResource');
// {!URLFOR($Resource.EMailServicesLead,EMailServicesLead/EmailServiceStaticResourceData.csv)}
Lead l = new Lead();
l.FirstName=email.fromName.substring(0,email.fromName.indexOf(''));
l.LastName=email.fromName.substring(email.fromName.indexOf(''));
l.Email = envelope.fromAddress;
myPlainText = email.plainTextBody;
l.Company = $Resource.EMailServicesLead.EmailServiceStaticResourceData.csv;
l.Status = $Resource.EMailServicesLead.EmailServiceStaticResourceData.csv;
l.LeadSource = $Resource.EMailServicesLead.EmailServiceStaticResourceData.csv;
l.Description = 'Mr.' + l.FirstName + l.LastName + ' enquired about the product Fin' + ' via' + ' Ëmail' + 'with body '+myPlainText;
Task[] newTask = new Task[0];
try { // Add a new Task to the Lead record we just found above.
List<Lead> vLed = [SELECT Id, Name, Email FROM Lead WHERE Email = :email.fromAddress LIMIT 1];
if(vLed.size()>0)
{ newTask.add(new Task(Description = myPlainText,
Priority = 'Normal',
Status = 'Inbound Email',
Subject = email.subject,
IsReminderSet = true,
ReminderDateTime = System.now()+1,
WhoId = vLed[0].Id));
insert newTask;
System.debug('New Task Object: ' + newTask );
}
else
{
insert l;
System.debug('Lead Created'+ l.Id);
}
}
catch (QueryException e) { System.debug('Query Issue: ' + e); }
return result;
}
}
You will need to parse the CSV to extract the data from it. To make this easier you can use this simple CSV parser class written by Nicolas Galler. Have a look through the code so you get how it works but all you will use is the constructor and the readLine() method that returns the list of values from each row of your CSV as Strings. You'll have to copy and paste the code from that link into an apex class named "SSSCsvReader" to be able to use it then in your code you will...
1. get your static resource CSV as a String
StaticResource sr = [SELECT ID, body FROM StaticResource WHERE Name = 'EMailServicesLead' LIMIT 1];
String csvData = sr.body.toString();
2. Create an instance of SSSCsvReader by calling the constructor passing in the CSV string as the argument(and optionally, the second argument of a delimiter but the comma is the default)
SSSCsvReader csvR = new SSSCsvReader(csvData);
3. Use the readLine() on your SSSCsvReader object to get CSV field values in a String list. (if Your file has a header row on it don't forget to remove this first by calling readLine() once before getting your data list).
csvR.readLine() //Removing header (optional)
String[] line = csvR.readLine();
if(line == null){return null} //No line in the file or there was no header so return
4. Assign each field in the returned list from readLine() to the corresponding field in your salesforce object.
Lead l = new Lead();
l.firstName = line[0]; //If your file is laid out like you said FirstName will be index [0]
l.lastName = line[1]; //LastName will be index [1]
.... //And so on
....
....
And there you have it a Lead object with fields from your CSV static resource file. If you have more rows on you CSV and want to create many leads than just set up a while loop that gets each line from the CSV using readLine(), create a new lead from this line and add it to a list.
I have written Solr Custom Search Component as described here
My goal is to update the query parameters, specifically to remove Unicode Quotes as early as possible in the pipeline.
However, after intercepting the request and editing the parameters, the request does not seem to update.
public void updateSolrRequest(SolrQueryRequest req) {
SolrParams params = req.getParams();
System.out.println( "params = " + req.getParamString());
String newQuery = params.get(CommonParams.Q);
newQuery = newQuery.toString().replaceAll("[A]","XXX");
ModifiableSolrParams newParams = new ModifiableSolrParams(params);
newParams.remove(CommonParams.Q);
newParams.add(CommonParams.Q, newQuery);
// all good to here, the next line should
// overwrite the old params with the new ones
// but it does not
req.setParams(newParams);
System.out.println("newQuery = " + newQuery);
System.out.println("newParams = " + newParams.get(CommonParams.Q));
System.out.println("updated req = " + req.getParamString());
}
Output
params = q=“A+Game+of+Thrones“&defType=dismax&q.alt=thrones&fq=Game&_=1548262845155
newQuery = “XXX Game of Thrones“
newParams = “XXX Game of Thrones“
updated req = q=“A+Game+of+Thrones“&defType=dismax&q.alt=thrones&fq=Game&_=1548262845155
The problem here is, that
public String getParamString() {
return origParams.toString();
}
is actually returning original params, which aren't affected by setParams called
/** Change the parameters for this request. This does not affect
* the original parameters returned by getOriginalParams()
*/
void setParams(SolrParams params);
You should use org.apache.solr.request.SolrQueryRequest#getParams to check your updated parameters.
I am using Akka.Net (v 1.3.2) and am trying to query the event journal for all events with a specific tag. I only want the events that exist at the time I query the journal. Inside an actor, I have the following code:
var readJournal = PersistenceQuery.Get(Context.System).ReadJournalFor<SqlReadJournal>(SqlReadJournal.Identifier);
var stream = readJournal.CurrentEventsByTag("The Tag Name", Offset.NoOffset());
var materializer = ActorMaterializer.Create(Context.System);
stream.RunForeach(envelope =>
{
// Do some stuff with the EventEnvelope
}, materializer).Wait();
This will successfully query the event journal. However, the problem is it will only return the first 100 events. I need all of them that match the query!
Question: How do I remove the limit/filter that exists when querying the event journal by tag name?
If you need it, here is my akka.persistence configuration:
var config = Akka.Configuration.ConfigurationFactory.ParseString(#"
akka.persistence {
journal {
plugin = ""akka.persistence.journal.sql-server""
sql-server {
class = ""Akka.Persistence.SqlServer.Journal.SqlServerJournal, Akka.Persistence.SqlServer""
connection-string = """ + connectionString + #"""
schema-name = dbo
table-name = __akka_EventJournal
metadata-table-name = __akka_Metadata
auto-initialize = on
}
}
snapshot-store {
plugin = ""akka.persistence.snapshot-store.sql-server""
sql-server {
class = ""Akka.Persistence.SqlServer.Snapshot.SqlServerSnapshotStore, Akka.Persistence.SqlServer""
connection-string = """ + connectionString + #"""
schema-name = dbo
table-name = __akka_SnapshotStore
auto-initialize = on
}
}
}"
);
There are two things to check out:
You can set the maximum number of messages returned in one query by setting up akka.persistence.query.journal.sql.max-buffer-size value (see: reference.conf).
Use readJournal.EventsByTag instead of readJournal.CurrentEventsByTag to get a continuous stream of events. Just keep in mind, that it won't complete by itself, but will live on waiting for new events to arrive. You can stop it explicitly i.e. by using KillSwitch.
I would like to have nancy rule that matches/captures all url segments after the initial match.
For example I would like to do this:
have a url like: /views/viewname/pageid/visitid/someother
and a rule like this:
Get["/views/{view}/{all other values}"] = parameters =>
{
string view = parameters.view;
List<string> listOfOtherValues = all other parameters..
return ...
};
listOfOtherValues would end up being:
pageid
visitid
someother
I would also like to do this for querystring parameters.
given the a url like: /views/viewname?pageid=1&visitid=34&someother=hello
then listOfOtherValues would end up being:
1
34
hello
Is this even possible with Nancy?
For your first problem you can use regular expression as well as simple names to define your capture groups. So you just define a catch all RegEx.
For your second, you just need to enumerate through the Request.Query dictionary.
Here is some code that demonstrates both in a single route.
public class CustomModule : NancyModule
{
public CustomModule() {
Get["/views/{view}/(?<all>.*)"] = Render;
}
private Response Render(dynamic parameters) {
String result = "View: " + parameters.view + "<br/>";
foreach (var other in ((string)parameters.all).Split('/'))
result += other + "<br/>";
foreach (var name in Request.Query)
result += name + ": " + Request.Query[name] + "<br/>";
return result;
}
}
With this in place you can call a URL such as /views/home/abc/def/ghi/?x=1&y=2 and get the output
View: home
abc
def
ghi
x: 1
y: 2
Note:
The foreach over Request.Query is support in v0.9+