I'm trying to load a filtered Wikidata dump with Anzograph using LOAD WITH 'global' <file:wdump-749.nt.gz> INTO GRAPH <WD_749>. The file exists; Anzograph gives out this error:
Error - At Turtle production subject=http://www.wikidata.org/entity/Q144> predicate=http://www.wikidata.org/prop/direct/P1319> file=wdump-749.nt.gz line=3229 details: -34000-01-01T00:00:00Z:Datum is not a datetime, use setting 'load_normalize_datetime' to patch bad data
I've set load_normalize_datetime=true in settings.conf and settings_anzograph.conf inside Anzograph's filesystem, restarted the server, but still can't load the dump. I get the exact same error.
load_normalize_datetime does not take a boolean. Change bad datetimes in loads to this value, e.g. 0001-01-01T00:00:00Z
So instead try setting:
load_normalize_datetime=0001-01-01T00:00:00Z
in your settings.conf, which worked for me on that specific file using the command you listed.
WD_749 has 38,131,614 statements, loaded in 372 seconds on my Thinkpad. It was relatively slow (102k triples per second) to load because it is a single file. If you break it up into smaller pieces (you can do this with the COPY command to dump the graph to a dir:/mydir/wdump-749.nt.gz) it will load in parallel (for me 114 seconds, 335k tps).
I'm building an SPA in AngularJS served by a Laravel (5.1) backend. Of late I've been encountering an annoying error, a server 500 or code 0 error which is abit hard to explain how it comes but let me try to may be someone will understand the dental formula of my problem.
When i start my AngularJS controller, I make several server calls (via independent $http calls from services) to retrieve information i might later need in the controller. For example,
Functions.getGrades()
.then(function(response)
{
$scope.grades = response.data;
});
Subjects.offered()
.then(function(response)
{
$scope.subjects = response.data;
});
Later on i pass these variables (grades or subjects) to a service where they are used for processing. However, these functions are randomly returning code 500 server errors after they run, and sometimes returning status code 0 after running. This happens in a random way and it is hard for me to point out the circumstances leading to their popping up. This leaves me with frequent empty Laravel-ised error screens like the ones shown below.
Anyone reading my mind?
Ok, after a suggestion given in a comment above that I check my Laravel log files (located in storage/logs/laravel.log- Laravel 5.1), i found out that the main error most of these times was this one: 'PDOException' with message 'SQLSTATE[HY000] [1044] Access denied for user ''#'localhost' to database 'forge'' in ..., plus another one that paraphrased something like No valid encrypter found. These were the key opener.
On reading another SO thread here, it said in part:
I solved, sometimes laravel not read APP_KEY in .ENV. And returns a value "SomeRandomString" (default is defined in config / app.php), and have the error "key length is invalid", so the solution is to copy the value of APP_KEY, to the value 'key 'in config / app.php, that's all! I solved!
That was exactly the issue! When loading the DB params from the .env to config/database.php, Laravel was sometimes unable to read the environment variables and went for the fallback default fallback options (forge for DB name and username and SomeRandomString for the APP_KEY). So, to solve this i just did as advised: copied the APP_KEY in .env to the config/app.php and edited the default DB parameters to the actual DB name and username/password I'm using. Just that and i was free from pollution. Hope someone finds this helpful.
UPDATE: the question i asked doesn't quite cover how deep i went in doing this upgrade! If you stumble on this, i hope the answer i pushed is useful to you
So, in an effort to get the "forum" plugin working, i've decided to upgrade cakephp from 1.3 to 2.1
(This was because the forum plugin uses something called CakeDC utils, and they are already up to cakephp 2.0, and surprise surprise, the 2.3 version of forum isn't clear which utils it works with...)
Ok, so I have run the cake upgrader magic thing, in accordance with the instructions found here:
http://book.cakephp.org/2.0/en/console-and-shells/upgrade-shell.html#upgrade-shell
And i ran the upgrader, and it said it did a bunch of things, and it looks like it did do a bunch of things.
Now when i visit my site, it is down - no response at all.
The error logs in apache read:
[Thu Apr 05 02:58:04 2012] [error] [client 173.45.125.112] PHP Fatal error: Can't find application core file. Please create /cake_install/app/config/core.php, and make sure it is readable by PHP. in /cake_install/cake/libs/configure.php on line 393
The reason that file doesn't exist is that it was changed to a capital "C" Config in the upgrade. What am i missing?
The given answer by mensch isn't quite what I need - I never touched the index.php file - so i will detail what I am trying here.
I am using this site as a guide, but with my own commentary too!
http://www.paulmarshall.us/blog/migrating-from-cakephp-13-to-cakephp-20
(i also found this: https://github.com/dereuromark/upgrade which i might have a shot at if the above tutorial fails. I'm not too keen on it though, I like super-exhaustive documents, not
Put `CakePlugin::loadAll();` in your app bootstrap to make sure this plugin is loaded
which to me is meaningless... can you specify the folder of the file? Any particular spot in the file to add the line? etc... i'll try and be particular about what i did!)
First up, i've noticed that the index.php that my cake was using was not the index.php that came with the cake2.1 code. Unsure why that is the case, but I copied the "new" 2.1 index.php across (yes, this is the app/webroot/index.php file) and now i have a new error:
Notice: Undefined index: Error in /cake_install/lib/Cake/Core/Configure.php on line 94 Notice: Undefined index: Exception in /cake_install/lib/Cake/Core/Configure.php on line 95 Fatal error: Class name must be a valid object or a string in /cake_install/lib/Cake/Error/ErrorHandler.php on line 126
(where cake_install is the folder where my cake app sits)
Working on this now...
AND now i see that the upgrade tool didn't upgrade the core.php file - found in app/Config (for 2.1, app/config for 1.3)... what did the upgrade tool really do? I've cut and paste the 2.1 version in (that is, from the zip file of the 2.1 code, i took that /app/Config/core.php and moved it into my /app/Config directory), and made sure the salt and security values were swapped across. That's it.
Still no luck. So now i have upgraded the database.php file (also in app/Config, or app/config) ~ I changed the
'driver' => 'mysqli',
to
'datasource' => 'Database/Mysqli',
Configuring files randomly is fun!
Ah, ok, so the upgrade to core and database changed the error message... that's good? Still unsure what their much vaunted "upgrade" script did... it just changed file names? Really?
Warning (2): file_put_contents(/cake_install/app/tmp/logs/error.log): failed to open stream: Permission denied [CORE/Cake/Log/Engine/FileLog.php, line 69]
Notice (1024): Please change the value of 'Security.cipherSeed' in app/Config/core.php to a numeric (digits only) seed value specific to your application [CORE/Cake/Utility/Debugger.php, line 810]
Fatal error: Class 'AppHelper' not found in /cake_install/lib/Cake/View/Helper/HtmlHelper.php on line 31
Ok, so i 777 the /cake_install/app/tmp/logs/ folder (chmod 777 -R folderHere) and
...guess i never upgraded the cipher seed. Ok, made up a new number! (What does the cipher seed do, you ask? Um, well, bing wasn't much help - i read the code. Seems to be used for encrypting/decrypting from cookies? Ok, i can live with that)
Bootstrap.php
I moved the bootstrap in the zip file to my version (thanks for little, upgrade script!). Haven't made any changes yet, I think i will need to to load plugins though... i'm sure that is well documented though (hah!)
So that leaves me with this errror:
Fatal error: Class 'AppHelper' not found in /cake_install/lib/Cake/View/Helper/HtmlHelper.php on line 31
And of course, the much vaunted cake upgrader has failed to do another upgrade. Check this page out:
http://book.cakephp.org/2.0/en/appendices/2-1-migration-guide.html
Ok, so now I have to make the files - just as they are in that page. The only trick was that, yes, it is in the app folder, and no, my app/View folder that cakePhp made didn't have a "Helper" subfolder... so for the AppHelper.php i had to make a Helper subfolder under /app/View/.
...
HEY! Now i get a background and a logo... and these:
Notice (8): Undefined variable: loggedIn [APP/pages/top_menu.html, line 3]
Notice (8): Undefined variable: html [APP/pages/top_menu.html, line 25]
Fatal error: Call to a member function link() on a non-object in /cake_install/app/pages/top_menu.html on line 25
Great. Some sort of cool new cake 2 feature that means that i cannot use $html? That seems... oddly appropriate, really.
Unbelievable. Now I need to change how $this->data works??? WHY?? WHY?? Why can't this mob work out that apis tend to be fixed contracts, not floating ideas... argh. That's ok, the most used cake line i have is now... wrong. I'm developing in scala or django from now on, junior languages suck.
Here is what i ran:
find . -type f -exec sed -i 's/$this->data/$this->request->data/g' {} \;
I went to the /app directory to do so. This 2.1... it better be seriously awesome... the amount of work everyone has to do to get it up. Is this, i assume, a normal day working with the junior languages?
Ah fantastic, the $html problem is caused because somebody didn't like lowercase? One can only wonder:
Undefined var when using Html Helper in CakePHP 2.0
And the solution is: (thanks to another post in SO)
find . -type f -exec sed -i 's/$html/$this->Html/g' {} \;
And naturally the same problem with $session -
find . -type f -exec sed -i 's/$session/$this->Session/g' {} \;
...
Things are coming along! Now I have the inscrutible error:
Controller class PagesController could not be found.
Oh for the love of... so i need to move the PagesController found here:
lib/Cake/Console/Templates/skel/Controller/PagesController.php
and put it here
app/Controller/
again, the Cake installer/upgrader seems to be asleep on the job.
And now I have
Helper class JavascriptHelper could not be found.
This isn't too bad - you won't find any references to "JavascriptHelper", but you will find references to "Javascript" in your "$helpers" array. The back-end adds in the word Helper for extra obfuscation. Anyway, i searched through my php files for any $helpers arrays, and found one, in the AppController(2.1) class.
I don't like nor understand helpers - javascript ones, anyway - so I never used them. I must have followed a tutorial blindly at some point and added it in... but i don't need it, so out it goes!
And now i need to clean up all the ctp files i had, where i used a "require" to ensure that the right html was used. Once more i will use the find command from above for this (to point them to the "View" and not "views" folder)
Sigh. Ok, no, it isn't that simple, is it? Where before I had :
require(app/View/Pages/man_front.html)
i now need:
require(/cake_folder/app/View/Pages/man_front.html)
(where the cake folder is where all my cake stuff is. Unsure why this is, but a simple find fixed it:
find . -type f -exec sed -i 's/app\/View\/Pages/\/cake_folder\/app\/View\/Pages/g' {} \;
(the \ will "depower" a /)
Ok, so having done all that and gotten your site to some sort of acceptable level, what next? Well, you might now find some db issues:
Warning (2): mysqli::mysqli() expects parameter 1 to be string, array given [CORE/Cake/Model/ConnectionManager.php, line 101]
Fatal error: Call to undefined method mysqli::getSchemaName() in /cake_install/lib/Cake/Model/Model.php on line 3383
Of course, no more mysqli support. Edited the database.php to use mysql instead (ie i just removed the ending "i")
(what is wrong with the peeps in cakephp??? API means "don't change everything if it will break existing code, ever." not "change! OBAMAAAAA!" ~ next upgrade, i'll push all my code to django or spring or scala. Ruddy nonsense.)
The next issue is the "Components" that i am using - i get a cake error saying:
Missing Plugin
Error: The application is trying to load a file from the Uploader plugin
Error: Make sure your plugin Uploader is in the app/Plugin directory and was loaded
<?php
CakePlugin::load('Uploader');
Loading all plugins: If you wish to load all plugins at once, use the following line in your app/Config/bootstrap.php file
CakePlugin::loadAll();
So... not only is all my code busted, but all the old plugins too? Nice.
The solution is to create an app/Plugin folder (yes, the magic upgrader doesn't do that, either) and then look at the plugins you had. Now! This is different from your components! I don't know how just yet - check out my question for a possible answer : Difference between a "component" and a "plugin" in cakephp 2.1? - but right now, we are just looking at the "PLUGIN" folder, and plugins in general. Looks like, in cake 1.3, the plugins were listed like this:
var $components = array('Uploader.Uploader',...);
that is, with a dot notation. I can live with that. In my case, I needed to get the latest version of the Uploader plugin (that, thankfully, was upgraded to be 2.x compatible!) and put it in the app/Plugin folder
so i got:
app
Plugin
Uploader
Config
Locale
<the other files here too!>
That seems to work right now, hooray!
The next error was:
Warning (2): Invalid argument supplied for foreach() [CORE/Cake/Utility/Set.php, line 1048]
Warning (2): array_multisort() [http://php.net/function.array-multisort]: Argument #1 is expected to be an array or a sort flag [CORE/Cake/Utility/Set.php, line 1087]
Warning (2): array_unique() expects parameter 1 to be array, null given [CORE/Cake/Utility/Set.php, line 1089]
Warning (2): Invalid argument supplied for foreach() [CORE/Cake/Utility/Set.php, line 1091]
Warning (2): Cannot modify header information - headers already sent by (output started at /cake_install/lib/Cake/Utility/Debugger.php:761) [CORE/Cake/Network/CakeResponse.php, line 395]
Ah, ok then. That is clear. I am glad I know what the error is in my code. Good error messaging!
Now I see, foolish me, to think that the utility Set::Sort would continue to work if the first array was empty! Truly, the cakephp people are standing, not on the shoulders of giants, but on their very foreheads.
It would seem that the new 2.1 upgrade does away with this shabby practice, so now i need a check before any Set:Sort calls to avoid if the array in question is empty or null. Fine. Easy enough, but... geeze.
I just threw in a
if (!empty($theArray)){
Set::sort($theArray,...);
}
and that fixed that.
Now I am getting FormHelper errors - it looks like $form in the view now becomes $this->Form, so i have another quick script to replace this:
find . -type f -exec sed -i 's/$form/$this->Form/g' {} \;
And now the next error I get is:
Fatal error: Unsupported operand types in /cake_install/lib/Cake/View/Helper/FormHelper.php on line 1804
And now they've done gone messed up the "select" method in the FormHelper - so before, where I had this:
echo $this->Form->select('treatment_id', $treatments, null, array('empty' => '-- select treatment to rate -- '));
i now have this:
echo $this->Form->select('treatment_id',$treatments, array('empty' => '-- select treatment to rate -- '));
And now one of my components - actually, the You_tube_loader_component - was broken. I need to modify it so that the header reads:
class YouTubeLoaderComponent extends Component {
function __construct(ComponentCollection $collection, $settings = array()) {
parent::__construct($collection, $settings);
...
}
}
Previously i had just defaulted the valeus of the uname and password - now i stick them in the "..." section instead of messing around with the mysterious "settings" array!
Next up is:
Warning (2): Illegal offset type [CORE/Cake/Model/Model.php, line 2689]
Warning (2): Illegal offset type [CORE/Cake/Model/Model.php, line 2665]
Soooo this is a problem that isn't picked up in cake 1.3, but is in 2.1
The issue is that in the controller action being referenced, there is a "find" call without a type paramter. That is, I had
$results = $this->User->find(array(...), array(...), null, false);
where i should have had:
$results = $this->User->find('first',array(
'conditions' => array("User.id" => ($this->Auth->user("id"))),
'fields' => array("User.confirmed")
)
);
Another issue I had was that, for whatever reason, the check:
$this->set('loggedIn', $this->Auth->user('id'));
did not work. Apparently (although am unsure how/why), cakePHP auth in 2.x will no longer store all the user's details to the session. I don't know which details it stores to the session, or how to modify them - looking into that! - but my code wasn't saving the "id" value, so the check above always set the loggedIn value to false.
To remedy, I modified the call to read:
$this->set('loggedIn', $this->Auth->loggedIn());
which worked fine for me.
Next problem is that the thios->auth->user('id') will return me sweet nothing after logging in a user. Fortunately I came across this post, that made things clearer:
http://ask.cakephp.org/questions/view/unable_to_access_logged-in_users_data_in_2_0
Amazingly, the geniuses at cake labs thought that changing the very way that logging in worked would help clear up some confusion. To be honest, the only thing I'm confused about is who was so confused about the fact that the auth component had all the user's details available after logging in. But fear not, they've cleared the confusion up and... removed the feature entirely.
So, what are you to do? The link above suggests getting all the user details, then shipping that into the auth login section. I don't particularly like this (because it is a stupid problem and i cannot believe anything so inelegant is needed), but it is what I have to do. Suggestions welcome!
So instead of the oh-so-confusing:
if ($this->Auth->login($this->request->data)) {
... //argh! I am so confused by that line above!!!1
i now have:
$user = $this->User->find('first', array(
'conditions' => array('User.email' => $this->request->data['User']['email'])
)
);
// Use the AuthComponent~Rs login action
if ($this->Auth->login($user['User'])) {
... //omg the extra check that the entry is in the db is just brilliant and efficient
isn't that just so much neater and less confusing? Good job, cakephp mavericks!
The next error is
Authorization adapter "controller" was not found.
Which is a bit vague...
Ah! Answered already on stack overflow here:
Authorization adapter was not found in cakePHP
And that was it! There might be other issues - god speed on them - but i didn't come across them, so i cannot help you any more.
You probably had a custom Cake path set in app/webroot/index.php. The directory structure for Cake 2.0 has changed slightly and normally this would be handled by the new code which automatically finds the right value for CAKE_CORE_INCLUDE_PATH. Also, make sure APP_DIR is set correctly in the same index.php file.
Make sure app/webroot/index.php contains the correct path to Cake, which is /path/to/cake/lib.
if (!defined('CAKE_CORE_INCLUDE_PATH')) {
define('CAKE_CORE_INCLUDE_PATH', DS.'path'.DS.'to'.DS.'cake'.DS.'lib');
}
Reverting back to the standard define('CAKE_CORE_INCLUDE_PATH', ROOT) might work as well, if the way Cake is set up is not too custom.
Here's my blog post on migrating from 1.3 to 2.x : http://www.forceflow.be/2011/11/12/migrating-from-cakephp-1-3-to-2-0/
For future reference, I've included the content of that post here. Enjoy!
Getting started
There are several ways to update, but for small to medium projects, I found it best to just unzip the new CakePHP 2 structure, and manually copy over your Controllers and Views to their new folders. Mind you that the folder structure has changed significantly. For instance, the controllers map is now Controller. Pay attention to where you copy the files – don’t worry about the filenames themselves, we’ll come to that later.
Make sure you update core.php and database.php (now located in (cakephp root)/app/Config) with the values for your application. Make sure to copy over the salt and cipherseed values from your old installation too, since CakePHP will continue whining when you use the default ones – it’s a security risk. Also, to get the database connection up and running again, the syntax was changed:
'driver' => 'mysqli'
becomes
'datasource' => 'Database/Mysql'
Bulk work: Upgrade via console
In order to rename all your app files to the new CakePHP file structure, there is a console command. New to CakePHP 2.0 is that each application now has its own console. Navigate to your app’s console (not the Console in /lib/Cake !) in /app/Console, make sure the Console command is executable by doing a chmod +x on it, and execute:
./cake upgrade all
This will rename all files to the new cake standard, and update all references within your PHP code.
Cleaning Up
Unfortunately, this is not the end. Chances are thin that your code will just run fine now. Have a look at the CakePHP 2.0 migration guide for more information. I’ll sum up the issues I’ve dealt with the most here:
In your views, you now have to address the Helper classes through the $this object. No more calling, for example
$html->link(
, it’s
$this->Html->link(
now, sir.
Also:
The JavascriptHelper is now called JsHelper.
The Auth component has changed significantly. The login() action is not automatically added when you use the component, you’ve got to manually specify it now. Check the new Auth component documentation for more info.
The AjaxHelper has been removed, along with the handy functions for AJAX-style searches, like observefield. I’ve cooked my own observefield using jQuery – use at your own risk. I serialise a form, which in this case holds a query input box which allows me to do a live-search-and-update for Reservations.
$this->Js->get('#query')->event('keyup', $this->Js->request(
array('controller' => 'sales','action' => 'searchReservations', $event['Event']['id']),
array(
'update' => '#view',
'async' => true,
'dataExpression' => true,
'method' => 'post',
'data' => $this->Js->serializeForm(array('isForm' => false, 'inline' => true)))
I had this same problem after using the Cake Shell Upgrade tool. While our web app is large, we never customized the index.php file.
Replacing our myapp/app/webroot/index.php with the newly downloaded CakePHP cakephp-2.3.4/app/webroot/index.php solved this issue for me.
I'm not sure why the CakePHP Upgrade Shell or it's instructions don't specify to change the index.php file, but as #bharal's lengthy and great answer shows, there is a lot more work to do since the Upgrade Shell doesn't do everything.