I have a script:
def tmn_file = ~/.*\.tmn/
def tmc_file = ~/.*\.tmc/
def newTerm = new Properties().with { props ->
new File(inputPath).eachFile(tmn_file) { file ->
file.withReader { reader ->
load( reader )
println "Read data from file $file:"
something read from file...
switch( props.ACTION ) {
case 'NEW':
do something...
}
switch( props.ACTION ) {
case 'CHANGE':
do something...
}
this script looking in directory with path inputPath file with extension tmn_file, which can containt ACTION - NEW or CHANGE.
Script works great, but i want to make another thing:
if file have extension *.tmn (tmn_file) - start only ACTION with NEW case
if file have extension *.tmc (tmc_file) - start only ACTION with CHANGE case
How i can realize decision?
Here is solution:
new Properties().with { props ->
new File(inputPath).eachFile(FileType.FILES) { file ->
file.withReader { reader ->
load(reader)
println "Read data from file $file:"
if (file.name.endsWith('tmn') & props.ACTION == 'NEW' || file.name.endsWith('tmc') & props.ACTION == 'CHANGE') {
// NEW mode
switch( props.ACTION ) {
case 'NEW':
...do someth...
break
// CHANGE mode
case 'CHANGE':
println "***CHANGE mode is on***"
...do someth...
break
default:
throw new RuntimeException("Unknown ACTION $props.ACTION")
}
} else {
if (file.name.endsWith('tmn') || file.name.endsWith('tmc')){
println "$file dont match for action $props.ACTION"
} else {
println "$file have wrong extension "}
}
}
}
}
Related
I'm trying to get the data in the array that came from another function(that function is extracting the data in the csv file) and when i tried calling the two fields from that array it shows an error that it is unidentified variables.
The $this->csv_process(); as shown on the function action() is the function that extracts the data from the csv file and stores it in an array which is successful since I tried checking it on var_dump();
I also named the two fields as $name and $email as shown below:
Function CSV_process()
public function csv_process()
{
/* variables for openning the csv file */
if (!in_array($extension, $allowed_ext)) {
$this->session->set_flashdata("message", "Sorry, CSV file only.");
} else {
if ($filesize > 0) {
$file = fopen($filename, "r");
$toWrite = array();
$error = false;
$col_size = 2;
$skip = 0;
while ($data = fgetcsv($file, 10000, ","))
{
$skip++;
if ($skip == 1) {
continue;
}
$numofcol = count($data);
if ($numofcol != $col_size ) {
$this->session->set_flashdata("message", "Column count exceeded or missing.");
} else {
$name1 = $data[0];
$name = str_replace("'", "''", $name1);
$email1 = $data[1];
$email = str_replace("'", "''", $email1);
$toWrite[] = [
'name' => $name,
'email' => $email
];
}
}
}
}
return $toWrite;
}
Function Action()
function action(){
$toWrite[] = $this->csv_process();
foreach ($toWrite as $arr) {
list($name, $email) = $arr;
//die(var_dump($arr));
$query = $this->db->query("SELECT * FROM import WHERE name ='$name' AND email = '$email'");
if ($query->num_rows() >= 1) {
} else {
if ($name == "" OR $email == "") {
} else {
if ((filter_var($email, FILTER_VALIDATE_EMAIL)) == FALSE ) {
} else {
$this->db->query("INSERT INTO import(name, email, created_date) VALUES('".$name."', '".$email."', '".date("Y-m-d h-i-s")."')");
$this->session->set_flashdata('message', 'SUCCESS YEAY');
redirect('Clean_csv/index');
}
}
}
$query->free_result();
}
}
Listing arrays doesn't seem to work for here, anyone knows how to extract the data array from $arr?
You don't need to extract the values. You can use each $arr in a bound query. It simplifies the syntax for the select query.
For inserting use CodeIgniter's insert() method. Again, the $arr can be used directly by adding the date to it before the insert is attempted.
I think this will work.
function action()
{
$toWrite[] = $this->csv_process();
foreach($toWrite as $arr)
{
$query = $this->db->query("SELECT * FROM import WHERE name=? AND email=?", $arr);
if($query->num_rows() >= 1)
{}
else
{
if($arr['name'] == "" OR $arr['email'] == "")
{}
else
{
if((filter_var($email, FILTER_VALIDATE_EMAIL)) == FALSE)
{}
else
{
$arr['created_date'] = date("Y-m-d h-i-s");
$this->db->insert("import", $arr);
$this->session->set_flashdata('message', 'SUCCESS YEAY');
//??? redirect('Clean_csv/index');
//Are you sure, you may still have more $arr in $toWrite to process - right?
}
}
}
$query->free_result();
}
}
You need to know what a terrible idea it is to repeatedly run database queries inside a loop. Even though you use free_result() it could be a massive drain on server resources. If your csv file has several thousand items you are severely stressing the database and the server.
I'm searching for something similar to Python's pickle. I want to do like this example:
https://www.quora.com/Is-there-a-way-to-keep-the-session-after-login-with-Selenium-Python
saving cookies
import pickle
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.quora.com')
# login code
pickle.dump(driver.get_cookies() , open("QuoraCookies.pkl","wb"))
loading cookies
import pickle
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://www.quora.com')
for cookie in pickle.load(open("QuoraCookies.pkl", "rb")):
driver.add_cookie(cookie)
I've tried many examples from the web without success. I saw the topics here as well.
In groovy case I'll :
Set<Cookie> allCookies = driver.manage().getCookies();
Variable allCookies should be serialized to file and deserialized later.
Java serialization is an option here. With credit to this post, here is an implementation:
import org.openqa.selenium.Cookie
def loadFile = { filename ->
def result = null
try {
def fis = new FileInputStream(filename)
def instream= new ObjectInputStream(fis)
result = (Set<Cookie>) instream.readObject()
instream.close()
} catch (Exception e) {
System.out.println(e)
}
return result
}
def saveFile = { set, filename ->
try {
def fos = new FileOutputStream(filename)
def out = new ObjectOutputStream(fos)
out.writeObject(set)
out.flush()
out.close()
} catch (IOException e) {
System.out.println(e)
}
}
and a sample driver program:
// ----------------------- main
if (args.size() >= 2) {
def action = args[0]
def file = args[1]
if (action == "write") {
def cookies = new HashSet<Cookie>()
cookies << new Cookie("Toronto", "Canada")
cookies << new Cookie("London", "UK")
cookies << new Cookie("Paris", "France")
saveFile(cookies, file)
} else if (action == "read") {
def cookies = loadFile(file)
cookies.each { cookie ->
println "${cookie.name} ${cookie.value}"
}
} else {
println "Unknown action: " + action
}
} else {
println "Usage: groovy CookieManager.groovy [read|write] filename"
}
Example usage (a) with all above code in the same script and (b) assuming that client-combined-3.0.0-beta2-nodeps.jar from here is on the classpath:
bash$ groovy CookieManager.groovy write set.dat
bash$ groovy CookieManager.groovy read set.dat
London UK
Toronto Canada
Paris France
I'm writing my own parser log for CakePHP.
I only need one thing: that is not written a log "message" (as a string), but a serialized array with various log information (date, type, line, stack traces, etc.).
But I don't understand what method/class I should rewrite, although I have consulted the APIs. Can you help me?
EDIT:
For now I do the opposite: I read the logs (already written) and I transform them into an array with a regex.
My code:
$logs = array_map(function($log) {
preg_match('/^'.
'([\d\-]+\s[\d:]+)\s(Error: Fatal Error|Error|Notice: Notice|Warning: Warning)(\s\(\d+\))?:\s([^\n]+)\n'.
'(Exception Attributes:\s((.(?!Request|Referer|Stack|Trace))+)\n)?'.
'(Request URL:\s([^\n]+)\n)?'.
'(Referer URL:\s([^\n]+)\n)?'.
'(Stack Trace:\n(.+))?'.
'(Trace:\n(.+))?(.+)?'.
'/si', $log, $matches);
switch($matches[2]) {
case 'Error: Fatal Error':
$type = 'fatal';
break;
case 'Error':
$type = 'error';
break;
case 'Notice: Notice':
$type = 'notice';
break;
case 'Warning: Warning':
$type = 'warning';
break;
default:
$type = 'unknown';
break;
}
return (object) af([
'datetime' => \Cake\I18n\FrozenTime::parse($matches[1]),
'type' => $type,
'error' => $matches[4],
'attributes' => empty($matches[6]) ? NULL : $matches[6],
'url' => empty($matches[9]) ? NULL : $matches[9],
'referer' => empty($matches[11]) ? NULL : $matches[11],
'stack_trace' => empty($matches[13]) ? (empty($matches[16]) ? NULL : $matches[16]) : $matches[13],
'trace' => empty($matches[15]) ? NULL : $matches[15]
]);
}, af(preg_split('/[\r\n]{2,}/', $logs)));
For now I do the opposite: I read the logs (already written) and with a regex I transform them into an array.
The problem is this is terribly expensive. and that it would be better to do the opposite: to write directly to the logs as a serialized array.
I think what you want to do is write your own LogAdapter.
You simply create a class ArrayLog (extends BaseLog) as mentioned in the docs and configure cakePHP to use it. Within the log function you append the information like $level, $message and $context to a file as an array. This will result in a log file with several arrays that then can be split.
That being said, I would suggest to log to the database and read it out instead of parsing.
Ok, that's it!
(note that this code is absolutely experimental, I have yet to test it properly)
One interesting thing that I want to do: for each log, write to the serialized file and also simultaneously in a plan file.
This allows me either to read logs as a plain text file, or they can be manipulated using the serialized file.
use Cake\Log\Engine\FileLog;
class SerializedLog extends FileLog {
protected function _getLogAsArray($level, $message) {
$serialized['level'] = $level;
$serialized['datetime'] = date('Y-m-d H:i:s');
//Sets exception type and message
if(preg_match('/^(\[([^\]]+)\]\s)?(.+)/', $message, $matches)) {
if(!empty($matches[2]))
$serialized['exception'] = $matches[2];
$serialized['message'] = $matches[3];
}
//Sets the exception attributes
if(preg_match('/Exception Attributes:\s((.(?!Request URL|Referer URL|Stack Trace|Trace))+)/is', $message, $matches)) {
$serialized['attributes'] = $matches[1];
}
//Sets the request URL
if(preg_match('/^Request URL:\s(.+)$/mi', $message, $matches)) {
$serialized['request'] = $matches[1];
}
//Sets the referer URL
if(preg_match('/^Referer URL:\s(.+)$/mi', $message, $matches)) {
$serialized['referer'] = $matches[1];
}
//Sets the trace
if(preg_match('/(Stack )?Trace:\n(.+)$/is', $message, $matches)) {
$serialized['trace'] = $matches[2];
}
$serialized['full'] = date('Y-m-d H:i:s').' '.ucfirst($level).': '.$message;
return (object) $serialized;
}
public function log($level, $message, array $context = []) {
$message = $this->_format(trim($message), $context);
$filename = $this->_getFilename($level);
if (!empty($this->_size)) {
$this->_rotateFile($filename);
}
$pathname = $this->_path . $filename;
$mask = $this->_config['mask'];
//Gets the content of the existing logs and unserializes
$logs = #unserialize(#file_get_contents($pathname));
if(empty($logs) || !is_array($logs))
$logs = [];
//Adds the current log
$logs[] = $this->_getLogAsArray($level, $message);
//Serializes logs
$output = serialize($logs);
if (empty($mask)) {
return file_put_contents($pathname, $output);
}
$exists = file_exists($pathname);
$result = file_put_contents($pathname, $output);
static $selfError = false;
if (!$selfError && !$exists && !chmod($pathname, (int)$mask)) {
$selfError = true;
trigger_error(vsprintf(
'Could not apply permission mask "%s" on log file "%s"',
[$mask, $pathname]
), E_USER_WARNING);
$selfError = false;
}
return $result;
}
}
I'm currently storing images within the webapp folder of my Lift project, which I know will cause problems in future.
val path = "src/main/webapp/files/"
And the code I'm using to save it:
case Full(file) =>
val holder = new File(path, "test.txt")
val output = new FileOutputStream(holder)
try {
output.write(file)
} finally {
output.close()
}
}
What I'm trying to do is save the to the server root in an easily manageable folder called files, so SERVER_ROOT/files outside of the project folder.
Firstly how would I access the path to the root of the server so I can save them there?
Secondly how would I serve these files from my app, so I can display them on a page?
Thanks in advance, any help much appreciated :)
You have to store file to exact place on filesystem according to absolute path. I have written this code and it works, so maybe it helps you:
def storeFile (file : FileParamHolder): Box[File] =
{
getBaseApplicationPath match
{
case Full(appBasePath) =>
{
var uploadDir = new File(appBasePath + "RELATIVE PATH TO YOUR UPLOAD DIR")
val uploadingFile = new File(uploadDir, file.fileName)
println("upload file to: " + uploadingFile.getAbsolutePath)
var output = new FileOutputStream(uploadingFile)
try
{
output.write(file.file)
}
catch
{
case e => println(e)
}
finally
{
output.close
output = null
}
Full(uploadingFile)
}
case _ => Empty
}
}
and this is my getBaseApplicationPath function which finds out absolute path of local machine (server or your devel PC):
def getBaseApplicationPath: Box[String] =
{
LiftRules.context match
{
case context: HTTPServletContext =>
{
var baseApp: String = context.ctx.getRealPath("/")
if(!baseApp.endsWith(File.separator))
baseApp = baseApp + File.separator
Full(baseApp)
}
case _ => Empty
}
}
when uploading an image to the server using cakephp
$this->Model->Behaviors->attach('ImageUpload', Configure::read('photo.files'));
photo uploaded successfully, and the database fields also
but shows following error instead of returning to index page.
Notice (8): Undefined index: class [CORE\cake\libs\model\behaviors\upload.php, line 104]
Notice (8): Undefined index: class [CORE\cake\libs\model\behaviors\upload.php, line 107]
Warning (2): Cannot modify header information - headers already sent by (output started at E:\umoorthy_105act10\projects\dev1base\core\cake\basics.php:111) [CORE\cake\libs\controller\controller.php, line 614]
wat to do?
Cake has already wrote where to look for a problem
Configure::read('photo.files')
do following to check if everything is ok
pr(Configure::read('photo.files'))
public function uploadFilesIphone($folder, $formdata, $replace , $itemId = null) {
// setup dir names absolute and relative echo "<pre>"; print_r($formdata); exit;
$folder_url = WWW_ROOT.$folder;
$rel_url = $folder; //echo
// create the folder if it does not exist
if(!is_dir($folder_url)) {
mkdir($folder_url);
}
// if itemId is set create an item folder
if($itemId) {
// set new absolute folder
$folder_url = WWW_ROOT.$folder.'/'.$itemId;
// set new relative folder
$rel_url = $folder.'/'.$itemId;
// create directory
if(!is_dir($folder_url)) {
mkdir($folder_url);
}
}
// list of permitted file types, this is only images but documents can be added
$permitted = array('image/gif','image/jpeg','image/pjpeg','image/png','application/octet-stream');
// loop through and deal with the files;
$key = array();
$value = array();
foreach($formdata as $key => $value)
{
if($key == is_array($value))
{
$filename = str_replace(".", $replace , $value['name']);
}
// replace spaces with underscores
// assume filetype is false
$typeOK = false;
// check filetype is ok
foreach($permitted as $type)
{
if($key == is_array($value))
{
if($type == $value['type'])
{
$typeOK = true;
break;
}
}
}
// if file type ok upload the file
if($typeOK) {
// switch based on error code
if($key == is_array($value))
{
switch($value['error'])
{
case 0:
// check filename already exists
if(!file_exists($folder_url.'/'.$filename))
{
// create full filename
$full_url = $folder_url.'/'.$filename;
$url = $rel_url.'/'.$filename;
// upload the file
if($key == is_array($value))
{
$success = move_uploaded_file($value['tmp_name'], $url);
}
}
else
{
// create unique filename and upload file
// ini_set('date.timezone', 'Europe/London');
$now = date('Y-m-d-His');
$full_url = $folder_url.'/'.$now.$filename;
$url = $rel_url.'/'.$now.$filename;
if($key == is_array($value))
{
$success = move_uploaded_file($value['tmp_name'], $url);
}
}
// if upload was successful
if($success)
{
// save the url of the file
$result['urls'][] = $url;
}
else
{
$result['errors'][] = "Error uploaded $filename. Please try again.";
}
break;
case 3:
// an error occured
$result['errors'][] = "Error uploading $filename. Please try again.";
break;
default:
// an error occured
$result['errors'][] = "System error uploading $filename. Contact webmaster.";
break;
}
}
elseif($value['error'] == 4)
{
// no file was selected for upload
$result['nofiles'][] = "No file Selected";
}
else
{
// unacceptable file type
$result['errors'][] = "$filename cannot be uploaded. Acceptable file types: gif, jpg, png.";
}
}
}
return $result;
}