Good evening :-)
I am learning Symfony(3) now, and I would like to use test for my classes. I've read that unittests shouldn't use databases, mocking its objects rather.
But despite this, I would like to create in setUp()/KernelTestCase database (e.q. MySQL) and read its content for file, next doing tests (simple -unittests), and purge at tearDown().
Is it possible do it with dumped MySQL file?
What is best (? lazy) way to do it?
I would rather to read prepared (dumped) SQL-like file then 'update:schema' from ORM-class. Where to put file in Symfony3? How read it and create database content?
Pls. help me.
I am using this for years:
Load tests data with the DoctrineFixtures bundle
Launch tests
Repeat! This is for me the most efficient as you only load your test data once; NOT for every tests. Of course you will have to think harder of your tests scenario.
For example, think about a crud test:
Test1:
Check that list has 1 item (loaded by fixtures)
Test2:
Create an item
Check that list has 2 items
Test3:
Delete an element
Check that the list has 1 item
So the tests must be executed in this order exactly. If you are loading fixtures between each test you don't have to take care of that order but it will make your tests slow.
I feel loading fixtures once is better because it will act as a user would normally do: create, delete, update items... And therefore you can check that there is no side effect between each action.
My load fixtures script:
#!/bin/bash
echo "##########################################################################"
echo "# Refresh data model, reload all reference data, load fixtures, #"
echo "# validate schema for the dev env. #"
echo "##########################################################################"
php bin/console doctrine:database:create --if-not-exists --env=dev
php bin/console doctrine:schema:drop --force --env=dev
php bin/console doctrine:schema:create --env=dev
php bin/console doctrine:schema:validate --env=dev
php bin/console doctrine:fixtures:load -n --env=dev
echo -e " --> DONE\n"
Or if you want to load a database for SQL files, use:
php bin/console doctrine:database:import db.sqb --env=dev
Instead of the load fixtures command.
Then, to launch the tests:
./bin/simple-phpunit --debug --verbose $1
$1 is an argument to specify the test suite to load. (main, front, API, backend...) which you can parameter in your phpunit.xml.dist file. (you can omit it to run all the tests)
My solution is:
This load all SQL files to MySQL TEST DB - defined in 'parameters_test.yml' and processing test and drops all DB tables before next test and again with next tests... It can be probably be done shorter and in more right way with command php bin/console doctrine:database:import ... as #Tokeeen.com said. Thank you for help.
// tests/AppBundle/Repository/BotsDbTest.php
<?php
use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase;
use Symfony\Component\Finder\Finder;
use Symfony\Component\Config\FileLocator;
use Symfony\Component\Yaml\Parser;
class BotsDbTest extends KernelTestCase
{
private $doctr;
private $db_cred;
private $db;
/**
* {#inheritDoc}
*/
protected function setUp()
{
$kernel = self::bootKernel();
$this->doctr = $kernel->getContainer()
->get('doctrine')
->getManager();
// for tests with loaded content
$this->db = new \AppBundle\Wiks\BotsDb();
// https://symfony.com/doc/current/bundles/extension.html
// get DB credientals from "parameters_test.yml":
$configDirectories = array( 'app/config' );
$locator = new FileLocator( $configDirectories );
$yamlUserFiles = $locator->locate( 'parameters_test.yml', null, false );
// https://davidegan.me/parse-yaml-in-php-using-symfony-yaml/
$yaml = new Parser();
$yaml_array = $yaml->parse( file_get_contents( $yamlUserFiles['0'] ) );
// needen DB is the second in Symfony - as database2 in file "parameters_test.yml":
$prefix_db = 'database2';
// looking for all keys with suffix: eq: 'database2_host'
$needed_sufix = [ 'host', 'port', 'name', 'user', 'password' ];
$this->db_cred = array();
foreach ( $yaml_array[ 'parameters' ] as $key => $value ) {
if ( strpos( $key, $prefix_db ) !== false ) {
foreach ( $needed_sufix as $needed_key ) {
if ( strpos( $key, $needed_key ) !== false ) {
$this->db_cred[ $needed_key ] = $value;
}
}
}
}
if ( count( $this->db_cred ) == count( $needed_sufix ) ) {
// check is all found
/*Array (
[host] => 127.0.0.1
[port] =>
[name] => db_name
[user] => user_name
[password] => ***
) */
$finder = new Finder();
// find and put into mysql all files as prepared content to tests
$finder->files()->name('*.sql');
foreach ( $finder->in( array( 'tests/dbcontent' ) ) as $file ) {
$shell_command = 'mysql --user='.$this->db_cred['user'].' --password='.$this->db_cred['password'];
$shell_command .= ' '.$this->db_cred['name'].'< '.$file->getRealPath();
shell_exec( $shell_command );
}
}
}
/**
* {#inheritDoc}
*/
protected function tearDown()
{
parent::tearDown();
// remoove DB content ( all tabels ):
$shell_command = 'mysqldump --user='.$this->db_cred['user'].' --password='.$this->db_cred['password'].' ';
$shell_command .= '--add-drop-table --no-data '.$this->db_cred['name'].' | ';
$shell_command .= 'grep -e \'^DROP \| FOREIGN_KEY_CHECKS\' | ';
$shell_command .= 'mysql --user='.$this->db_cred['user'].' --password='.$this->db_cred['password'].' '.$this->db_cred['name'];
shell_exec( $shell_command );
$this->doctr->close();
$this->doctr = null; // avoid memory leaks
}
/** tests, tests, tests...
*
*/
public function test_getBots()
{
$res = $this->db->getBots( $this->doctr );
$this->assertEquals(5, count( $res ));
[...]
Helpful links:
How to remove all MySQL tables from the command-line without DROP database permissions?
https://davidegan.me/parse-yaml-in-php-using-symfony-yaml/
https://symfony.com/doc/current/components/finder.html
https://symfony.com/doc/current/bundles/extension.html
Related
I am running the following Powershell Script against a list of SQL Instances to return Instance information, including users and roles. I want to use Powershell to do this as I'm collating the data and will import into another system which will do some analysis.
I created the following script which creates an XML file output for each instance found. The script works great (yes, it's probably clunky and an awful way to do this, but I'm learning so please feel free to give me a shove in the right direction), however for one of the servers with a few hundred SQL logins I get an overflow message appear on screen, the XML file is not closed correctly and as a result I can't import the results into my analysis system.
I would like either:
Ideas on what could be causing the overflow. For reference, the output XML that crashes out is 2.5MB in size, with approx 39,000 lines in the XML output before the overflow occurs
[OR]
Another way to get this output - CSV is an option but I don't know enough how to output this - can anyone provide tips?
Thank you in advance
#Input file is a plain text file with the name of each of the instances listed in it
$InputFile="C:\Tasks\SQL\Permissions\in\Instances.txt"
$OutputFolder="\\networkdrive\sharedfolder\"
Function GetDBUserInfo($Dbase)
{
if ($dbase.status -eq "Normal") # ensures the DB is online before checking
{$users = $Dbase.users | where {$_.login -eq $SQLLogin.name} # Ignore the account running this as it is assumed to be an admin account on all servers
foreach ($u in $users)
{
if ($u)
{
$XmlWriter.WriteStartElement("Login")
$XmlWriter.WriteElementString('DBName', $dbase.name)
$XmlWriter.WriteElementString('LoginName', $SQLLogin.name)
$XmlWriter.WriteStartElement('Login_Roles')
$DBRoles = $u.enumroles()
foreach ($role in $DBRoles)
{
$XmlWriter.WriteElementString('Role', $Dbase.name)
}
$XmlWriter.WriteEndElement()#Login_Roles
#Get any explicitly granted permissions
$XmlWriter.WriteStartElement('Login_Permissions')
$XmlWriter.WriteElementString('Instance', $svr.name)
$XmlWriter.WriteElementString('DBName', $dbase.name)
$XmlWriter.WriteElementString('LoginName', $SQLLogin.name)
foreach($perm in $Dbase.EnumObjectPermissions($u.Name))
{
$XmlWriter.WriteElementString('Permissions', $perm.permissionstate.tostring() + " " + $perm.permissiontype.tostring() + " on " + $perm.objectname.tostring() + " in " + $DBase.name.tostring())
}
$XmlWriter.WriteEndElement() #Login_Permissions
$XMLWriter.WriteEndElement() #Login
} # Next user in database
}
#else
#Skip to next database.
}
}
#Main portion of script start
[reflection.assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo") | out-null #ensure we have SQL SMO available
foreach ($SQLsvr in get-content $InputFile) # read the instance source file to get instance names
{
$svr = new-object ("Microsoft.SqlServer.Management.Smo.Server") $SQLsvr
#Cycle through each instance and write the instance information to the file
#Output file is base folder for each of the text files (which will be named for the instance)
$OutputFile = $svr.name
$OutputFile = $OutputFolder+$OutputFile.Replace("\", "-")+".xml"
# get an XMLTextWriter to create the XML
$XmlWriter = New-Object System.XMl.XmlTextWriter($OutputFile,$Null)
# choose a pretty formatting:
$xmlWriter.Formatting = 'Indented'
$xmlWriter.Indentation = "4"
# write the header
$xmlWriter.WriteStartDocument()
# set XSL statements
$XLSPropText="type='text/xsl' href='style.xsl'"
$xmlWriter.WriteProcessingInstruction("xml-stylesheet", $XSLPropText)
# create root element "instances" and add some attributes to it
$xmlWriter.WriteStartElement("Root")
$XmlWriter.WriteStartElement("Instance")
$XmlWriter.WriteElementString("SQLInstance", $svr.name)
$XmlWriter.WriteElementString("SQLVersion", $svr.VersionString)
$XmlWriter.WriteElementString("Edition", $svr.Edition)
$XmlWriter.WriteElementString("LoginMode", $svr.loginmode)
$XmlWriter.WriteEndElement #instance
$SQLLogins = $svr.logins
foreach ($SQLLogin in $SQLLogins)
{
#Iterate through each login, writing the details into the login details
#$XmlWriter.WriteComment("Login Details")
$xmlWriter.WriteStartElement("Logins")
$XmlWriter.WriteElementString("InstanceName", $svr.Name)
$XmlWriter.WriteElementString("LoginName", $SQLLogin.Name)
$XmlWriter.WriteElementString("LoginType", $SQLLogin.LoginType)
$XmlWriter.WriteElementString("Created", $SQLLogin.CreateDate)
$XmlWriter.WriteElementString("DefaultDatabase", $SQLLogin.DefaultDatabase)
$XmlWriter.WriteElementString("Disabled", $SQLLogin.IsDisabled)
$SQLRoles = $SQLLogin.ListMembers()
If ($SQLRoles)
{ $XmlWriter.WriteElementString("ServerRole", $SQLRoles) }
else
{ $XmlWriter.WriteElementString("ServerRole", "Public") }
If ( $SQLLogin.LoginType -eq "WindowsGroup" )
{ #get individuals in any Windows domain groups
$XmlWriter.WriteStartElement("WindowsLogins")
$XmlWriter.WriteElementString("InstanceName", $svr.name)
$XmlWriter.WriteElementString("Login", $SQLLogin.name)
try {
$ADGRoupMembers = get-adgroupmember $SQLLogin.name.Split("\")[1] -Recursive
foreach($member in $ADGRoupMembers)
{ $XmlWriter.WriteElementString("Account", $member.name.tostring() + "(" + $member.SamAccountName.tostring() +")") }
}
catch
{
#Sometimes there are 'ghost' groups left behind that are no longer in the domain, this highlights those still in SQL
$XmlWriter.WriteElementString("Account", "Unable to locate group " + $SQLLogin.name.Split("\")[1] + " in the AD Domain")
}
$XmlWriter.WriteEndElement()
}
#Check the permissions in the DBs the Login is linked to.
If ($SQLLogin.EnumDatabaseMappings())
{
$XmlWriter.WriteStartElement('Permissions')
$XmlWriter.WriteElementString('InstanceName', $svr.name)
$xmlwriter.WriteElementString('Login', $SQLLogin.name)
foreach ( $DB in $svr.Databases)
{
try {
GetDBUserInfo($DB)
}
catch
{
echo $_.Exception|format-list -force
}
} # Next Database
$XmlWriter.WriteEndElement()
}
Else
{
$XmlWriter.WriteStartElement('Permissions')
$XmlWriter.WriteElementString('InstanceName', $svr.name)
$xmlwriter.WriteElementString('Login', $SQLLogin.name)
$XmlWriter.WriteElementString('Permissions', 'No Permissions')
$XmlWriter.WriteEndElement()
}
$xmlWriter.WriteEndElement() #End Logins element
}
}
# close the "machines" node:
$xmlWriter.WriteEndElement() #root node
# finalize the document:
$xmlWriter.WriteEndDocument()
$xmlWriter.Flush()
$xmlWriter.Close()
When running, error message is:
OverloadDefinitions
--------------------
void WriteEndElement()
void WriteEndElement()
No other error or message is given. The file does get written but is incomplete.
when user select the file then in server side file extension will be removed or rename and uploaded it to the server using php codeigniter library
This should remove your file extension
<?php
$var = "testfile.php";
$explode = explode( '.', $var );
array_pop( $explode);
$var = implode( '.', $explode );
var_dump( $var );
As for uploading you will need to read the manual on file uploading
http://www.codeigniter.com/user_guide/libraries/file_uploading.html
file name can be changed using the following codes:
$your_given_name = time().rand().$_FILES["userfiles"]['name'];
$config['file_name'] = $your_given_name;
for changing extension you can use:
if ($this->upload->do_upload('file_name')) {
$file_data=$this->upload->data();
$new_name_by_you='anything'.$file_data['file_ext'];
$new_path=$file_data['file_path'].$new_name_by_you;
rename($file_data['full_path'], $new_path);
}
rename() is a php built in function. for details please visit http://php.net/manual/en/function.rename.php
I recently just had the same issue, and found it really annoying that even when you specified a file_name in code igniter it would append the file extension. Which i did not want.
Blinkydamo's link is not helpfull, since that is the link i was following in the first place and makes no mention of this dilema, although it does demontrate that it is not possible idrectly through Code Igniter - by ommission of the answer.
I haven't tried md asif rahman's method, but it looks sound.
Alternatively you can just forget about using Code Igniter's upload function, and fall back onto PHPs standard one, which is what I have done :
move_uploaded_file($_FILES['imageFile']['tmp_name'], '/path/'.$filename);
where $filename is the exact name of the file including extension, therefore if none is specified - it wont have one.
it is so easy
change code in
libraries/Upload.php
public function set_filename($path, $filename)
{
if ($this->encrypt_name === TRUE)
{
$filename = md5(uniqid(mt_rand()));
}
if ($this->overwrite === TRUE OR ! file_exists($path.$filename))
{
return $filename;
}
$filename = str_replace($this->file_ext, '', $filename);
$new_filename = '';
for ($i = 1; $i < $this->max_filename_increment; $i++)
{
if ( ! file_exists($path.$filename.$i))
{
$new_filename = $filename.$i;
break;
}
}
if ($new_filename === '')
{
$this->set_error('upload_bad_filename', 'debug');
return FALSE;
}
else
{
return $new_filename;
}
}
I need a cron job in one of my older web site (CakePHP 1.3).
I made the following steps:
created shell in /somepath/public_html/app/vendors/shells/valute.php
class ValuteShell extends Shell {
var $uses = array('Valute');
function main() {
$this->Valute->cacheGetKonverterData();
}
}
created BASH script and save this to my vendors folder as cakeshell (/somepath/public_html/app/vendors/cakeshell)
#!/bin/bash
TERM=dumb
export TERM
cmd="cake"
while [ $# -ne 0 ]; do
if [ "$1" = "-cli" ] || [ "$1" = "-console" ]; then
PATH=$PATH:$2
shift
else
cmd="${cmd} $1"
fi
shift
done
$cmd
changed the permissions on the this file to 777.
called cronjob like this:
0 0 * * * /somepath/public_html/app/vendors/cakeshell valute -cli /usr/bin -console /somepath/public_html/cake/console -app /somepath/public_html/app
Unfortunately cronjob does not work. Why?
When I call: http://www.somedomain.net/valutes/save_valute
everything works fine.
<?php
class ValutesController extends AppController {
var $name = 'Valutes';
var $uses = array('Valute');
function save_valutes()
{
$this->layout = null; // turn off the layout
$this->Valute->cacheGetKonverterData();
}
}
?>
cakeshell: cmd should have the PATH of the Cake core's cake file
cmd="/somepath/public_html/lib/Cake/Console/cake"
crontab:
0 0 * * * /somepath/public_html/app/vendors/cakeshell valute -app /somepath/public_html/app >> var/log/crontab
I understand there are MANY ways to do all of this, but trying to do it the best way.
I have created the db parameters, dns, dbh, sth, sql and generally quite happy with the result up to ... well ... the result part.
<?php
// db parameters
$dbhost = "localhost";
$dbname = "x";
$dbuser = "y";
$dbpass = "z";
// driver invocation (dsn is short for data source name)
$dsn = "mysql:host=$dbhost;dbname=$dbname";
// create db object (dbh is short for database handle)
$dbh = new PDO($dsn, $dbuser, $dbpass);
// execution of database query (sth is short for statement handle)
$sql = "SELECT * FROM a_aif_remaining";
$sth = $dbh->prepare($sql);
$sth->execute();
NOT SURE WHAT TO PUT BELOW.... (A) or (B)
I just want to present a simple array of the data. One row from the table per line.
Option A
echo $_POST['fieldname1'];
echo $_POST['fieldname2'];
echo $_POST['fieldname3'];
Option B
while ($rows = $sth->fetch(PDO::FETCH_ASSOC)) {
echo $row[fieldname1],'<br>';
}
AND I AM CONFIDENT WITH THE ENDING
$dbh = NULL;
?>
Any advise would be GREATLY appreciated.
UPDATED CODE: (Produces nothing on the page)
<?php
// db parameters
$dbhost = "localhost";
$dbname = "theaudit_db1";
$dbuser = "theaudit_user";
$dbpass = "audit1999";
$dsn = "mysql:host=$dbhost;dbname=$dbname"; // driver invocation (dsn is short for Data Source Name)
try {
$dbh = new PDO($dsn, $dbuser, $dbpass); // connect to new db object (dbh is short for Database Handle)
$dbh->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); // set the PDO error mode to enable exceptions
$dbh->setAttribute(PDO::ATTR_EMULATE_PREPARES, false); // set the PDO emulate prepares to false
// execute query to database (sth is short for Statement Handle)
$sql = "SELECT * FROM a_aif_remaining";
$sth = $dbh->prepare($sql);
$sth->execute();
$data = $sth->fetchAll(PDO::FETCH_ASSOC);
$dbh = NULL;
}
catch(PDOException $e)
{
echo $e->getMessage();
}
?>
Though I can't get what's the connection between A anb B, I can answer the
I just want to present a simple array of the data. One row from the table per line.
question.
$sql = "SELECT * FROM a_aif_remaining";
$sth = $dbh->prepare($sql);
$sth->execute();
$data = $sth->fetchAll(PDO::FETCH_ASSOC);
where $data is a sought-for array.
The problem with your updated code is simple - you arent echo'ing your data out. You need to add something like..
foreach($data as $arKey=>$dataRow){
foreach($dataRow as $arKey=>$rowField){
echo $rowField.','; //concat with a ',' to give csv like output
}
echo '<br>'; //to get to next line for each row (may want to trim the last ','
}
I am also confused by the reference to $_POST. It is true both are associate arrays but that does not mean that the $_POST option is viable - the data would only be available in the $_POST if you put it there (eg $_POST = $data) which would be pointless. Or if you had posted the data from somewhere else. Neither seem to fit what you are asking so I would forget about the $_POST and just figure out how you access your multi dimensional $data array. There is endless tut's on this subject. Try using
var_dump($data)
to see whats inside that should help you visualise what is going on.
NOTE: in option B you are not correctly concatenating or referencing your array it should be:
while ($rows = $sth->fetch(PDO::FETCH_ASSOC)) {
echo $rows[fieldname1].'<br>'; //$row doesnt exist its $rows and you use . to concat not ,
}
Ah yes and probably better to use unset rather than setting $dbh to equal null
unset($dbh);
Im using the Musicbrainz cpan module to look up an album but Im having a few issues trying to decipher the output I recieve. I used data::Dumper to have a look at it, and it appears to be a hash or array of some sort but when I try to check the type I run into problems.
my $ws = WebService::MusicBrainz::Release->new();
my $response = $ws->search({ TITLE => 'ok computer' });
if (ref($response) eq "REF" || ref($response) eq "SCALAR" || ref($response) eq "ARRAY" || ref($response) eq "HASH" || ref($response) eq "CODE" || ref($response) eq "GLOBE")
{
print "\n What sort of thing is it? \n";
}
Thanks
It's a WebService::MusicBrainz::Response object.
use WebService::MusicBrainz::Release;
my $ws = WebService::MusicBrainz::Release->new();
my $response = $ws->search({ TITLE => 'ok computer' });
my $release = $response->release(); # grab first one in the list
print $release->title(), " (", $release->type(), ") - ", $release->artist()->name(), "\n";
Like already said, it is a WebService::MusicBrainz::Response object.
You can retrieve multiple results with accessing the release_list() which gives an array of WebService::MusicBrainz::Response::Release objects.
use WebService::MusicBrainz::Release;
my $ws = WebService::MusicBrainz::Release->new();
my $response = $ws->search({ TITLE => 'ok computer' });
my #releaselist = $response->release_list();
foreach my $release ( #releaselist ) {
print $release->title(), " - ", $release->artist()->name(), "\n";
}
However, that perl module is somewhat unmaintained and the XML Web service Version 1 it is using is deprecated.
You better use Version 2 of the Web Service.
With python-musicbrainzngs there is a python module available that uses the new (next generation scheme) Web Service.