When I query Google AppEngine's datastore using PHP(through Quercus) and the low-level data-access API for an entity, I get an error that the entity doesn't exist, even though I've put it in the datastore previously.
The specific error is "com.caucho.quercus.QuercusException: com.google.appengine.api.datastore.DatastoreService.get: No entity was found matching the key: Test(value1)"
Here's the relevant code -
<?php
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.EntityNotFoundException;
import com.google.appengine.api.datastore.Key;
import com.google.appengine.api.datastore.KeyFactory;
import com.google.appengine.api.datastore.PreparedQuery;
import com.google.appengine.api.datastore.Query;
$testkey = KeyFactory::createKey("Test", "value1");
$ent = new Entity($testkey);
$ent->setProperty("field1", "value2");
$ent->setProperty("field2", "value3");
$dataService = DatastoreServiceFactory::getDatastoreService();
$dataService->put($ent);
echo "Data entered";
try
{
$ent = $dataService->get($testkey);
echo "Data queried - the results are \n";
echo "Field1 has value ".$ent->getProperty("field1")."\n";
echo "Field2 has value ".$ent->getProperty("field2")."\n";
}
catch(EntityNotFoundException $e)
{
echo("<br/>Entity test not found.");
echo("<br/>Stack Trace is:\n");
echo($e);
}
And here's the detailed stack-trace - link.
This same code runs fine in Java (of course after changing the syntax). I wonder what's wrong.
Thanks.
I have found the solution to my problem. It was caused by missing dependencies and I solved it by using the prepackaged PHP Wordpress application available here.
One thing is to be noted. The package overlooked a minor issue in that all files other than the src/ directory need to be in a war/ directory which stays alongside the src/ directory (this as per appengine conventions as mentioned on its documentation). So I organized the files thus myself, put the above PHP file in the war/ directory, and it's working fine on the appengine.
Related
I want to have additional files which can be imported in preprocess.py file
but i am not able to import these directly.
My directory looks like this:
Want to import from helper_functions directory into preprocess.
I tried to add this in setup.py file but it did not work.
package_data={"pipelines.ha_forecast.helper_functions": ["*.py"]},
One thing which kind of worked was to add this folder in input like this:
inputs = [
ProcessingInput(source=f'{project_name}/{module_name}/helper_functions',
destination="/opt/ml/processing/input/code/helper_functions"),
]
But this was putting the required files in some other directory which I was not able to import anymore.
What is standard way of doing this?
You have to specify the source_dir. Within your script then you can import the modules as you normally do.
source_dir (str or PipelineVariable) – Path (absolute, relative or an
S3 URI) to a directory with any other training source code
dependencies aside from the entry point file (default: None). If
source_dir is an S3 URI, it must point to a tar.gz file. Structure
within this directory are preserved when training on Amazon SageMaker.
Look at the documentation in general for Processing (you have to use FrameworkProcessor and not the specific ones like SKLearnProcessor).
P.S.: The answer is similar to that of the question "How to install additional packages in sagemaker pipeline".
Within the specified folder, there must be the script (in your case preprocess.py), any other files/modules that may be needed, and also eventually the requirements.txt file.
The structure of the folder then will be:
BASE_DIR/
|─ helper_functions/
| |─ your_utils.py
|─ requirements.txt
|─ preprocess.py
Within your preprocess.py, you will call the scripts in a simple way with:
from helper_functions.your_utils import your_class, your_func
So, your code becomes:
from sagemaker.processing import FrameworkProcessor
from sagemaker.sklearn import SKLearn
from sagemaker.workflow.steps import ProcessingStep
from sagemaker.processing import ProcessingInput, ProcessingOutput
BASE_DIR = your_script_dir_path
sklearn_processor = FrameworkProcessor(
estimator_cls=SKLearn,
framework_version=framework_version,
instance_type=processing_instance_type,
instance_count=processing_instance_count,
base_job_name=base_job_name,
sagemaker_session=pipeline_session,
role=role
)
step_args = sklearn_processor.run(
inputs=[your_inputs],
outputs=[your_outputs],
code="preprocess.py",
source_dir=BASE_DIR,
arguments=[your_arguments],
)
step_process = ProcessingStep(
name="ProcessingName",
step_args=step_args
)
It's a good practice to keep the folders for the various steps separate for each and don't create overlaps.
My spring boot app uses logback to log messages in json format. The app is configured to use consolelogappender (stdout).When the logs appear in stackdriver, they appear as textPayload instead of jsonPayload. Is it possible to write message to jsonPayload field in stackdriver using logback? If not, what are my options to log in json format?
Based on this Github Link it seems the issue all log entries are seen as textpayload. It has been added as a Feature Request but we do not have an ETA on when it will be available.
I'm not entirely sure if an alternative exist as Logback seems to be giving extensive log information, but if you are able to use the Stackdriver Logging Client instead, you could format the entry in order to get your object as a JsonPayLoad, although you will have specify most of the log categories yourself which can be an extra amount of work.
The easy way to do this, is to implements the transformation of TextPayload(JSON Format) to JSONPayload on the LoggingEnhacer
Check this answer How to use Stackdriver Structured Logging in App Engine Flex Java environment
It is possible via google-cloud-logging-logback library.
However, please note following (from https://cloud.google.com/logging/docs/structured-logging):
Note: message is saved as textPayload if it is the only field remaining
after the Logging agent moves the other special-purpose fields and
detect_json wasn't enabled; otherwise message remains in jsonPayload.
detect_json is not applicable to managed logging environments like
Google Kubernetes Engine.
To add more data to json add an enhancer. Example:
import ch.qos.logback.classic.spi.ILoggingEvent;
import com.google.cloud.logging.LogEntry;
import com.google.cloud.logging.Payload;
import com.google.cloud.logging.logback.LoggingEventEnhancer;
import java.util.HashMap;
public class EventEnhancer implements LoggingEventEnhancer {
#Override
public void enhanceLogEntry(
LogEntry.Builder logEntry,
ILoggingEvent e
) {
HashMap<String, Object> map = new HashMap<>();
map.put("thread", e.getThreadName());
map.put("context", e.getLoggerContextVO().getName());
map.put("logger", e.getLoggerName());
Payload.JsonPayload payload = logEntry.build().getPayload();
map.putAll(payload.getDataAsMap());
logEntry.setPayload(
Payload.JsonPayload.of(map)
);
}
}
Configuration:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration>
<configuration scan="true">
<appender name="CLOUD" class="com.google.cloud.logging.logback.LoggingAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<log>application.log</log>
<redirectToStdout>true</redirectToStdout>
<resourceType>gae_app</resourceType>
<loggingEventEnhancer>EventEnhancer</loggingEventEnhancer>
<flushLevel>INFO</flushLevel>
</appender>
<root level="INFO">
<appender-ref ref="CLOUD"/>
</root>
</configuration>
I'm very new to Fat-Free and Backbone.js. I've been searching and reading articles and searching and reading articles trying to find a way to route to individual PHP files containing the database communications. The code below works, and I can use it, but it seems hackish. Is there a way to call an external PHP file (in the server/models/ directory) and a specific method from the $f3-route(...) line?
<?php
// File: /index.php
define("PATH",1);
$f3 = require('server/fatfree/lib/base.php');
$uri = explode('/', $_SERVER["REQUEST_URI"]);
require_once "server/models/{$uri[PATH]}.php";
$f3->route('GET /hello/#file', 'HelloModel->doSomething');
$f3->route('GET /project/#file', 'ProjectModel->doSomething');
$f3->route('GET /book/#file', 'BookModel->doSomething');
$f3->run();
?>
Thanks a lot for your advice.
You should add the server/models directory to the autoloader of F3 using the autoloader feature.
$f3->set('AUTOLOAD','server/models/');
That way, the required source files of your classes will be loaded on demand. However, note that the files must be named the same as your class, i.e. class Foo has to be defined in foo.php or Foo.php. The case of the filename does not matter.
I'm sure I am missing something embarrassingly obvious here, but as I am following Tipfy's Hello World tutorial, I'm getting nothing but Werkzeug's 404 page whenever I try to load one of the URLs defined in my urls.py file (in /project/app/apps/hello_world).
I've ensured that 'apps.hello_world' is in the list of installed apps in the 'apps_installed' dictionary in config.py. Everything else seems to be in order, and I've read & followed the instructions in the tutorial and Getting Started guide carefully, but I can't see why Tipfy isn't detecting the rules I defined in get_rules() in my hello_world app's urls.py file.
Here is the code from that file:
from tipfy import Rule
def get_rules(app):
rules = [
Rule('/', endpoint='hello/world', handler='handlers.HelloWorldHandler'),
Rule('/hello-jinja', endpoint='hello/jinja', handler='handlers.HelloJinjaHandler'),
Rule('/hello-json', endpoint='hello/json', handler='handlers.HelloJsonHandler'),
Rule('/hello-ajax', endpoint='hello/ajax', handler='handlers.HelloAjaxHandler'),
]
return rules
Any help anyone can offer is greatly appreciated - thanks!
Nevermind. I solved it. Although it was nowhere in the Tipfy documentation that I could find, and despite the fact that I'm still a solid newbie with Python, I guess I should have known that my 'hello_world' directory in the apps directory needed an __init__.py file in order to be treated as a module
Since in log4j javadoc is
WARNING: This version of JDBCAppender is very likely to be completely replaced in the future. Moreoever, it does not log exceptions.
What should I do to log to a database?
If you are looking for a database appender which not only works, but also supports connection pooling, is maintained and properly documented, than consider logback's DBAppender.
Ironically enough, the warning in the javadocs about removing JDBCAppender in future versions of log4j was written by me.
You can use an alternative appender, but really Log4j 1.2 is going to be around and standard for a long time. They developed DBAppender as part of their receivers companions, which isn't officially released, but you can download the source code and get your own going as well.
Unless the issue of not logging exceptions bothers you, JDBCAppender is just fine. Any further upgrade to 2.0 is going to be more radical than just changing JDBCAppender (if 2.0 happens), so I wouldn't worry about using it, despite the warning. They clearly don't have a solid roadmap or timeline to introducing a new version, and 1.2.15 was released in 2007.
**log4j.properties file**
# Define the root logger with appender file
log4j.rootLogger = DEBUG, DB
# Define the DB appender
log4j.appender.DB=org.apache.log4j.jdbc.JDBCAppender
# Set JDBC URL
log4j.appender.DB.URL=jdbc:mysql://localhost/log
# Set Database Driver
log4j.appender.DB.driver=com.mysql.jdbc.Driver
# Set database user name and password
log4j.appender.DB.user=root
log4j.appender.DB.password=root
# Set the SQL statement to be executed.
log4j.appender.DB.sql=INSERT INTO actionlg(user_id, dated, logger, level, message) values('%X{userId}',' %d{yyyy-MM-dd-HH-mm}','%C','%p','%m')
# Define the layout for file appender
log4j.appender.DB.layout=org.apache.log4j.PatternLayout
**Java Class**
Log4jExamples.java
import java.sql.*;
import java.io.*;
import org.apache.log4j.Logger;
import org.apache.log4j.MDC;
public class Log4jExample {
/* Get actual class name to be printed on */
static Logger log = Logger.getLogger(Log4jExample.class.getName());
public static void main(String[] args)throws IOException,SQLException{
log.error("Error");
MDC.put("userId", "1234");
}
}
**libs required**
- mysql-connector-java-3.1.8-bin.jar
- log4j-1.2.17.jar