I am new to Gatling. So excuse me on my ignorance
I have the following code.
For users from 1 to 10, all the files in src/test/resources/data are posted using http BUT
I want to post one http request per file
Eg below there are 10 files in src/test/resources/data. So when I say
rampUsersPerSec(1) to (10) during (60) one file should be posted per user.
Any leads are appreciated. Thank you
val httpConf = http.baseUrl("http://localhost:8082/")
.header("Accept", "application/json")
val files = getListOfFiles("src/test/resources/data")
val scenarioOne = scenario("Post feature extractor")
.exec(
files.map( fileName => runFeatureExtractor(fileName))
)
setUp(
scenarioOne.inject(
nothingFor(5),
rampUsersPerSec(1) to (10) during (60)
)
).protocols(httpConf)
def runFeatureExtractor(fileName: String) = {
exec(
http("Extract features")
.post("feature")
.body(StringBody(fromFile(fileName).mkString)).asJson
.check(status.is(200))
)
}
I am currently working on a google cloud project in free trial mode. I have cron job to fetch the data from a data vendor and store it in the data store. I wrote the code to fetch the data couple of weeks ago and it was all working fine but all of sudden , i started receiving error "DeadlineExceededError: The overall deadline for responding to the HTTP request was exceeded" for last two days. I believe cron job is supposed to timeout only after 60 minutes any idea why i am getting the error?.
cron task
def run():
try:
config = cron.config
actual_data_source = config['xxx']['xxxx']
original_data_source = actual_data_source
company_list = cron.rest_client.load(config, "companies", '')
if not company_list:
logging.info("Company list is empty")
return "Ok"
for row in company_list:
company_repository.save(row,original_data_source, actual_data_source)
return "OK"
Repository code
def save( dto, org_ds , act_dp):
try:
key = 'FIN/%s' % (dto['ticker'])
company = CompanyInfo(id=key)
company.stock_code = key
company.ticker = dto['ticker']
company.name = dto['name']
company.original_data_source = org_ds
company.actual_data_provider = act_dp
company.put()
return company
except Exception:
logging.exception("company_repository: error occurred saving the company
record ")
raise
RestClient
def load(config, resource, filter):
try:
username = config['xxxx']['xxxx']
password = config['xxxx']['xxxx']
headers = {"Authorization": "Basic %s" % base64.b64encode(username + ":"
+ password)}
if filter:
from_date = filter['from']
to_date = filter['to']
ticker = filter['ticker']
start_date = datetime.strptime(from_date, '%Y%m%d').strftime("%Y-%m-%d")
end_date = datetime.strptime(to_date, '%Y%m%d').strftime("%Y-%m-%d")
current_page = 1
data = []
while True:
if (filter):
url = config['xxxx']["endpoints"][resource] % (ticker, current_page, start_date, end_date)
else:
url = config['xxxx']["endpoints"][resource] % (current_page)
response = urlfetch.fetch(
url=url,
deadline=60,
method=urlfetch.GET,
headers=headers,
follow_redirects=False,
)
if response.status_code != 200:
logging.error("xxxx GET received status code %d!" % (response.status_code))
logging.error("error happend for url: %s with headers %s", url, headers)
return 'Sorry, xxxx API request failed', 500
db = json.loads(response.content)
if not db['data']:
break
data.extend(db['data'])
if db['total_pages'] == current_page:
break
current_page += 1
return data
except Exception:
logging.exception("Error occured with xxxx API request")
raise
I'm guessing this is the same question as this, but now with more code:
DeadlineExceededError: The overall deadline for responding to the HTTP request was exceeded
I modified your code to write to the database after each urlfetch. If there are more pages, then it relaunches itself in a deferred task, which should be well before the 10 minute timeout.
Uncaught exceptions in a deferred task cause it to retry, so be mindful of that.
It was unclear to me how actual_data_source & original_data_source worked, but I think you should be able to modify that part.
crontask
def run(current_page=0):
try:
config = cron.config
actual_data_source = config['xxx']['xxxx']
original_data_source = actual_data_source
data, more = cron.rest_client.load(config, "companies", '', current_page)
for row in data:
company_repository.save(row, original_data_source, actual_data_source)
# fetch the rest
if more:
deferred.defer(run, current_page + 1)
except Exception as e:
logging.exception("run() experienced an error: %s" % e)
RestClient
def load(config, resource, filter, current_page):
try:
username = config['xxxx']['xxxx']
password = config['xxxx']['xxxx']
headers = {"Authorization": "Basic %s" % base64.b64encode(username + ":"
+ password)}
if filter:
from_date = filter['from']
to_date = filter['to']
ticker = filter['ticker']
start_date = datetime.strptime(from_date, '%Y%m%d').strftime("%Y-%m-%d")
end_date = datetime.strptime(to_date, '%Y%m%d').strftime("%Y-%m-%d")
url = config['xxxx']["endpoints"][resource] % (ticker, current_page, start_date, end_date)
else:
url = config['xxxx']["endpoints"][resource] % (current_page)
response = urlfetch.fetch(
url=url,
deadline=60,
method=urlfetch.GET,
headers=headers,
follow_redirects=False,
)
if response.status_code != 200:
logging.error("xxxx GET received status code %d!" % (response.status_code))
logging.error("error happend for url: %s with headers %s", url, headers)
return [], False
db = json.loads(response.content)
return db['data'], (db['total_pages'] != current_page)
except Exception as e:
logging.exception("Error occured with xxxx API request: %s" % e)
return [], False
I would prefer to write this as a comment, but I need more reputation to do that.
What happens when you run the actual data fetch directly instead of
through the cron job?
Have you tried measuring a time delta from the start to the end of
the job?
Has the number of companies being retrieved increased dramatically?
You appear to be doing some form of stock quote aggregation - is it
possible that the provider has started blocking you?
I have a probleme with PHPMailer .
First i used it without FTP , but all email was SPAM .
So i tried to use my FTP login and now all email are not junk .
But problem ... my attachement ( .rar and . zip ) doest work .
i tried with txt files and its ok ... i'm becoming crazy
require('class.phpmailer.php');
$mail = new PHPMailer();
$mail->CharSet = "UTF-8";
$mail->SMTPDebug = 1;
$mail->IsSMTP();
$mail->Host = "r******";
$mail->Port = "465";
//usually the port for TLS is 587, for SSL is 465 and non-secure is 25
$mail->SMTPSecure = "ssl";
//TLS, SSL or delete the line
$mail->SMTPAuth = true;
$mail->Username = 's*****';
$mail->Password = '##';
$mail->From = 'sales#*****';
$mail->FromName = '****';
$mail->AddAddress("$payer_email", "$payer_business_name");
$mail->Subject = "SUBJET";
$mail->Body = " MY MESSAGE";
if( $custom == 1 )
{
$mail->AddAttachment('test.txt');
}
if( $custom == 2 )
{
$mail->AddAttachment('test2.txt');
}
$mail->Send() ;
}
}
fclose ($fp);
}
i use if and else because its a paypal button with param . ( if 1 i send this file by mail )
this is working with txt files . but not with RAR or ZIP files ... No email send
any idea ?
thanks a lot
i've a very specific problem, but i'm a niewbie with zend framework so i don't have idea of how exctly this db adapter works as a configuration, but i've already made a db connection with the default adapter of zend, and it was successful. Now i've to set two different database connections for two different db in the same application. So i've taken my application.ini and i've written the following lines:
;connessione al db
resources.db.adapter = pdo_mssql
resources.db.params.host = "ip"
resources.db.params.username = user
resources.db.params.password = pwd
resources.db.params.dbname = NAME
resources.db.isDefaultTableAdapter = true
resources.db.params.pdoType = dblib
;connessione al db1
resources.db1.adapter = pdo_mssql
resources.db1.params.host = "ip"
resources.db1.params.username = user
resources.db1.params.password = pwd
resources.db1.params.dbname = NAME
resources.db1.isDefaultTableAdapter = false
resources.db1.params.pdoType = dblib
then i went to my action controller and i wrote:
$db = Zend_Registry::get ( 'db' );
$result = $db->fetchRow("SELECT [Sell-to Customer No_] FROM dbo.SyncroPlanningTable WHERE id='".$id);
$rag_soc=$result->{"Sell-to Customer No_"};
$db1 = Zend_Registry::get ( 'db1' );
$result1 = $db1->fetchRow("SELECT [No_],[Name],[Address],[City],[Contact],[Name],[Phone] FROM `dbo.SOS$Customer` WHERE No_ = '".$rag_soc."'");
The error i'm getting is the following:
Fatal error: Uncaught exception 'Zend_Application_Bootstrap_Exception' with message 'Unable to resolve plugin "db1";
UPDATE:
My bootstrap.php is:
$resource = $this->getPluginResource ( "db" );
$db = $resource->getDbAdapter ();
$db->setFetchMode ( Zend_Db::FETCH_OBJ );
Zend_Db_Table_Abstract::setDefaultAdapter ( $db );
Zend_Registry::set ( "db", $db );
How can i change it? it is not mentioned in the manual page you gave me.
resources.db refers to Zend_Application_Resource_Db, so "db" here is not a variable name.
You should use Zend_Application_Resource_Multidb to support multiple database connections:
http://framework.zend.com/manual/1.12/en/zend.application.available-resources.html#zend.application.available-resources.multidb
Your code is expecting the DB adapters to be in the registry, so you need to grab them from the multiDB resource and store them:
$multiDB = $this->getPluginResource('multidb');
Zend_Registry::set('db1', $multiDB->getDb('db1');
Zend_Registry::set('db2', $multiDB->getDb('db2');
also, this line:
Zend_Db_Table_Abstract::setDefaultAdapter ( $db );
can be removed, as you're specifying the default adapter in the application.ini.
Here are my code:-
//Sending mail
if ($this->Session->read('Enrollment.personalinfo_language') == 'English') {
$language = "english";
$subject = "Thank you for submitting your enrollment request to Apna Energy.";
} else {
$language = "spanish";
$subject = "Gracias por enviar su solicitud de inscripci?n a Apna Energy.";
}
$details = $this->Session->read('Enrollment');
$details['plan_name'] = $product['Product']['name'];
$details['rate'] = $plan_rate;
$details['term'] = $product['Term']['term'];
$this->Email->sendAs = 'html';
$this->Email->from = 'Apna Energy <contact#apnaenergy.com>';
$this->Email->to = $this->Session->read('Enrollment.personalinfo_first_name') . ' ' . $this->Session->read('Enrollment.personalinfo_last_name') . '<' . $this->Session->read('Enrollment.personalinfo_email') . '>';
$this->Email->bcc = array('my#mail.com');
$this->Email->subject = $subject;
$this->set('details', $details);
if ($this->Session->read('Enrollment.personalinfo_language') == 'English') {
$template = "enrollment_confirmation";
} else {
$template = "enrollment_confirmation";
}
$this->Email->template = $template;
$this->Email->send();
My problem is if customer fill form they are receiving mail in his/her spam folder.. customer's mail id in "to".. and my mail id in "BCC" for me mail are coming fine in my inbox folder..
I followed two URL but they didn't work out for me..
Cakephp emails going to spam
How do you make sure email you send programmatically is not automatically marked as spam?
Guide me in right direction..
Thanks!!
by using SMTP
with normal PHP your server must be configured properly which is not easy to do as a beginner (MX records need to match the servers ip etc).
So just always stick to SMTP as mailing gateway and you will be fine.
PS: I don't think it has anything to do with your code in general, although it is not very beautiful. for instance: you should cast the array you read from the session to avoid notices thrown:
$details = (array)$this->Session->read('Enrollment');