Datanucleus JPA 2 Level 2 cache in Google AppEngine - google-app-engine

What am I missing in the following configuration that my Datanucleus JPA 2 Level 2 cache is not using Google App Engine Memcache service? I am using the GAE 1.7.2 SDK.
In the persistence.xml:
<persistence-unit name="transactions-optional">
<provider>org.datanucleus.api.jpa.PersistenceProviderImpl</provider>
<properties>
<property name="datanucleus.NontransactionalRead" value="true"/>
<property name="datanucleus.NontransactionalWrite" value="true"/>
<property name="datanucleus.ConnectionURL" value="appengine"/>
<property name="datanucleus.appengine.datastoreReadConsistency" value="EVENTUAL" />
<property name="javax.persistence.query.timeout" value="5000" />
<property name="datanucleus.datastoreWriteTimeout" value="10000" />
<property name="datanucleus.singletonEMFForName" value="true"/>
<property name="datanucleus.cache.level2.cacheName" value="someName"/>
<property name="datanucleus.cache.level2.type" value="javax.cache"/>
</properties>
</persistence-unit>
Entity annotations contain:
#Entity
#Cacheable(true)
My WEB-INF/lib looks like this:
-rw-r--r-- 1 501 20 27M Oct 3 16:13 appengine-api-1.0-sdk-1.7.2.jar
-rw-r--r-- 1 501 20 3.3M Oct 3 16:13 appengine-api-labs.jar
-rw-r--r-- 1 501 20 5.0M Oct 3 16:13 appengine-endpoints.jar
-rw-r--r-- 1 501 20 6.8K Oct 3 16:13 appengine-jsr107cache-1.7.2.jar
-rw-r--r-- 1 501 20 45K Oct 3 16:13 asm-4.0.jar
-rw-r--r-- 1 501 20 309K Oct 3 16:13 datanucleus-api-jdo-3.1.0-m3.jar
-rw-r--r-- 1 501 20 246K Oct 3 16:13 datanucleus-api-jpa-3.1.0-m3.jar
-rw-r--r-- 1 501 20 331K Oct 3 16:13 datanucleus-appengine-2.1.0-final.jar
-rw-r--r-- 1 501 20 1.6M Oct 3 16:13 datanucleus-core-3.1.0-m5.jar
-rw-r--r-- 1 501 20 112K Oct 3 16:13 geronimo-jpa_2.0_spec-1.0.jar
-rw-r--r-- 1 501 20 5.2M Oct 3 16:13 gwt-servlet.jar
-rw-r--r-- 1 501 20 196K Oct 3 16:13 jdo-api-3.0.1.jar
-rw-r--r-- 1 501 20 7.9K Oct 3 16:13 jsr107cache-1.1.jar
-rw-r--r-- 1 501 20 15K Oct 3 16:13 jta-1.1.jar
In the WEB-INF/lib directory, running:
for i in *.jar; do echo $i; jar -tf $i | grep Level2; done
yields:
appengine-api-1.0-sdk-1.7.2.jar
appengine-api-labs.jar
appengine-endpoints.jar
appengine-jsr107cache-1.7.2.jar
asm-4.0.jar
datanucleus-api-jdo-3.1.0-m3.jar
datanucleus-api-jpa-3.1.0-m3.jar
datanucleus-appengine-2.1.0-final.jar
datanucleus-core-3.1.0-m5.jar
org/datanucleus/cache/WeakLevel2Cache.class
org/datanucleus/cache/Level2Cache$PinnedClass.class
org/datanucleus/cache/AbstractLevel2Cache.class
org/datanucleus/cache/SoftLevel2Cache.class
org/datanucleus/cache/Level2Cache.class
org/datanucleus/cache/NullLevel2Cache.class
org/datanucleus/cache/JavaxCacheLevel2Cache.class
geronimo-jpa_2.0_spec-1.0.jar
gwt-servlet.jar
jdo-api-3.0.1.jar
jsr107cache-1.1.jar
jta-1.1.jar
which shows that org.datanucleus.cache.JavaxCacheLevel2Cache is present in datanucleus-core-3.1.0-m5.jar. Yet, when I try to access entities that have been persisted before this cache was enabled, I get the following error on the server side.
org.datanucleus.exceptions.NucleusUserException: Level 2 Cache "javax.cache" is registered to use class "org.datanucleus.cache.JavaxCacheLevel2Cache" yet this is not found. Please check your CLASSPATH and plugin specification.
Further down the exception stack trace, I find:
Caused by: java.lang.NoClassDefFoundError: javax/cache/Caching
at org.datanucleus.cache.JavaxCacheLevel2Cache.<init>(JavaxCacheLevel2Cache.java:63)
Now, that is clear but where can I find this javax.cache.Caching? Which JAR should I need to include in the classpath? A cursory Google search does not immediately answer my question.
Thanks.

Google "memcached" uses an old version of what will become the standard "javax.cache". DataNucleus 3.0 supports only this old version (with properties as you have). DataNucleus 3.1 supports both the old version of javax.cache and the most recent. To use the old version (i.e GAE memcached) with DN 3.1 you need to set the property datanucleus.cache.level2.type to jcache. See http://www.datanucleus.org/products/accessplatform_3_1/jpa/cache.html#jcache
Why you're using DataNucleus version 3.1.0-m3/m5 I've no idea when there are many releases since then!

Related

File permissions log4j2 with RollingFile

I have problem with file permissions with RollingFile and log4j2 configuration.
Application is creating log file as follows:
-rwxr-xr-x. 1 centos centos 10379932 Feb 8 13:55 application.log
after rolling files read permissions for others are missing. I'm not using any permissions setting with log4j
-rw-r-----. 1 root root 524 Feb 9 13:42 application.log
-rw-r-----. 1 root root 462705 Feb 9 10:24 application.log.1.gz
umask is: 0022
my log4j2.xml
<RollingFile name="test_app" fileName="${sys:application.home}/logs/application.log" ignoreExceptions="false" filePattern="${sys:application.home}/logs/application.log.%i.gz">
<PatternLayout pattern="%d{dd-MMM-yyyy HH:mm:ss,SSS} - %X{userId} - %X{sessionId} %-5p %c{1}:%L - %m%n"/>
<Policies>
<SizeBasedTriggeringPolicy size="10MB"/>
</Policies>
<DefaultRolloverStrategy max="50"/>
</RollingFile>
I have 2 environments and on the second one permissions are as expected (application and log4j2 configuration is the same)
Where can I search the problem?

How to use Selenium with Firefox browser when I have no display

I am newly assigned to this shared hosting (ssh access only). Have no sudo or yum facility. I am trying to scrape some data from a dynamically loaded website [So, can't use scrapy or bs]. When I am using Selenium, it gives an error:
/home/sanelywr/voyager-bot/app.py:28: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Firefox(options=options, executable_path=gecko_path)
Traceback (most recent call last):
File "/home/sanelywr/voyager-bot/app.py", line 28, in <module>
driver = webdriver.Firefox(options=options, executable_path=gecko_path)
File "/home/sanelywr/virtualenv/voyager-bot/3.9/lib/python3.9/site-packages/selenium/webdriver/firefox/webdriver.py", line 180, in __init__
RemoteWebDriver.__init__(
File "/home/sanelywr/virtualenv/voyager-bot/3.9/lib/python3.9/site- packages/selenium/webdriver/remote/webdriver.py", line 275, in __init__
self.start_session(capabilities, browser_profile)
File "/home/sanelywr/virtualenv/voyager-bot/3.9/lib/python3.9/site- packages/selenium/webdriver/remote/webdriver.py", line 365, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/home/sanelywr/virtualenv/voyager-bot/3.9/lib/python3.9/site- packages/selenium/webdriver/remote/webdriver.py", line 430, in execute
self.error_handler.check_response(response)
File "/home/sanelywr/virtualenv/voyager-bot/3.9/lib/python3.9/site- packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 1
When checking the geckodriver.log it says,
1653393333249 geckodriver INFO Listening on 127.0.0.1:33457
1653393333256 mozrunner::runner INFO Running command: "/home/sanelywr/voyager-bot/firefox/firefox" "--marionette" "--headless" "--no-sandbox" "-headless" "--disable-dev-shm-usage" "--remote-debugging-port" "37442" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "/tmp/rust_mozprofilei8tj67"
(firefox:28332): Gtk-WARNING **: 07:55:33.285: Locale not supported by C library.
Using the fallback 'C' locale.
**Error: no display specified**
Apparently, I have no screen on this server as I have checked by the following command:
$ echo $DISPLAY
It displays nothing.
I have already set the browser option to headless mode [within the python code].
And, I have also done the following:
$ export MOZ_HEADLESS = 1
And yes, I cannot use the virtual screen like xvfb because even when I am able to pip install the related wrapper, I still cannot install xvfb because I don't have sudo or yum [and cannot find a .tar.bz2 or tar.gz file].
How can I run the selenium browser without having to open the browser on any screen altogether? [Tried PhantomJS() too, even that's not working]
My python script :
import os
from credentials import API_KEY, API_SECRET, ACCESS_TOKEN, ACCESS_TOKEN_SECRET
#from pyvirtualdisplay.xephyr import XephyrDisplay
#from pyvirtualdisplay import Display
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
import tweepy
auth = tweepy.OAuthHandler(API_KEY, API_SECRET)
auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)
api = tweepy.API(auth)
options = Options()
options.add_argument("--headless")
options.add_argument('--no-sandbox')
options.headless= True
#options = webdriver.ChromeOptions()
#options.add_argument('--no-sandbox')
options.add_argument("--disable-dev-shm-usage")
#options.binary_location = '/home/sanelywr/voyager-bot'
#driver = webdriver.Chrome(executable_path='./chromedriver', options=options)
#driver = webdriver.PhantomJS()
gecko_path = '/home/sanelywr/voyager-bot/geckodriver'
options.binary=FirefoxBinary('/home/sanelywr/voyager-bot/firefox/firefox')
#display = XephyrDisplay()
#display = Display(visible=0, size=(800, 600))
#display.start()
driver = webdriver.Firefox(options=options, executable_path=gecko_path)
url=""
driver.get(url)
#<-----------Some code for extracting some elements from the page----------------->
#driver.quit()
#display.stop()
My script folder:
drwxr-xr-x 4 sanelywr sanelywr 4.0K Jan 24 2016 phantomjs-2.1.1-linux-x86_64
-rw-rw-r-- 1 sanelywr sanelywr 23M Jan 24 2016 phantomjs-2.1.1-linux-x86_64.tar.bz2
drwxr-xr-x 7 sanelywr sanelywr 4.0K Apr 7 2016 firefox
-rw-rw-r-- 1 sanelywr sanelywr 50M Apr 11 2016 firefox-45.0.2.tar.bz2
-rwxr-xr-x 1 sanelywr sanelywr 8.3M Apr 6 11:54 geckodriver
drwxr-xr-x 2 sanelywr sanelywr 4.0K May 23 02:09 public
drwxr-xr-x 2 sanelywr sanelywr 4.0K May 23 02:09 tmp
-rw-r--r-- 1 sanelywr sanelywr 145 May 23 02:09 passenger_wsgi.py
-rw-r--r-- 1 sanelywr sanelywr 326 May 23 02:30 credentials.py
-rw-r--r-- 1 sanelywr sanelywr 0 May 23 02:31 temp.txt
-rw-r--r-- 1 sanelywr sanelywr 16 May 23 02:36 requirements.txt
drwxrwxr-x 2 sanelywr sanelywr 4.0K May 23 06:29 __pycache__
-rw-r--r-- 1 sanelywr sanelywr 8.3M May 24 05:26 geckodriver-v0.31.0-linux64.tar
-rw-r--r-- 1 sanelywr sanelywr 1.9K May 24 07:24 app.py
-rw-rw-r-- 1 sanelywr sanelywr 4.8K May 24 07:55 geckodriver.log

How to change user/group uid/gid in LXC container and affecting the used files/directorys

In a fresh Debian 11 container I installed urbackup.
In the container the uid/gid from urbackup is 107/115.
I needed to bind mounted a zfs dataset to store the actual backups..
Therefore I changed in lxc the urbackup user/group.
root#urBackup:~# usermod -u 1005 urbackup
root#urBackup:~# groupmod -g 1005 urbackup
It seems though that some directory's stil had old uid/gid
root#urBackup:~# ls -al /var/urbackup
total 108012
drwxr-xr-x 3 nobody 115 4096 Apr 19 05:54 .
drwxr-xr-x 12 root root 4096 Apr 19 05:46 ..
-rwxr-x--- 1 nobody 115 35044777 Apr 19 05:47 UrBackupUpdate.exe
-rwxr-x--- 1 nobody 115 40 Apr 19 05:47 UrBackupUpdate.sig
-rwxr-x--- 1 nobody 115 102 Apr 19 05:47 UrBackupUpdata.sig2
Is there a easy way to change also the directory's and files belonging to a certain user/group.
Thanks for reading
Guy

How to install Aerospike on Mac?

I tried following: https://www.aerospike.com/docs/operations/install/vagrant/mac/
and did following:
mkdir aerospike-vm
cd aerospike-vm
vagrant init aerospike/aerospike-ce
This creates Vagrantfile in the same directory.
Next I try:
vagrant up
Getting error:
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'aerospike/aerospike-ce' could not be found. Attempting to find and install...
default: Box Provider: virtualbox
default: Box Version: >= 0
The box 'aerospike/aerospike-ce' could not be found or
could not be accessed in the remote catalog. If this is a private
box on HashiCorp's Atlas, please verify you're logged in via
`vagrant login`. Also, please double-check the name. The expanded
URL and error message are shown below:
URL: ["https://atlas.hashicorp.com/aerospike/aerospike-ce"]
Error: The requested URL returned error: 404 Not Found
What is the proper procedure to install Aerospike on Mac?
Thanks
I recorded this few months ago along with two other videos in this set. See if this youtube video helps you out: https://www.youtube.com/watch?v=qm42c0juam4&list=PLGo1-Ya-AEQDa32hFggyB0yIIOldxUFwv&index=3
Here is the output from my run:
Administrators-MacBook-Pro-4:aerospike-vm piyush$ ls -al
total 8
drwxr-xr-x 4 piyush staff 128 Apr 9 2018 .
drwxr-xr-x+ 54 piyush staff 1728 Jan 18 10:56 ..
drwxr-xr-x 3 piyush staff 96 Apr 9 2018 .vagrant
-rw-r--r-- 1 piyush staff 3029 Apr 9 2018 Vagrantfile
Administrators-MacBook-Pro-4:aerospike-vm piyush$ mv Vagrantfile Vagrantfile_old
Administrators-MacBook-Pro-4:aerospike-vm piyush$ vagrant init aerospike/aerospike-ce
==> vagrant: A new version of Vagrant is available: 2.2.3!
==> vagrant: To upgrade visit: https://www.vagrantup.com/downloads.html
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
Administrators-MacBook-Pro-4:aerospike-vm piyush$ ls -al
total 16
drwxr-xr-x 5 piyush staff 160 Jan 31 14:13 .
drwxr-xr-x+ 54 piyush staff 1728 Jan 18 10:56 ..
drwxr-xr-x 3 piyush staff 96 Apr 9 2018 .vagrant
-rw-r--r-- 1 piyush staff 3029 Jan 31 14:13 Vagrantfile
-rw-r--r-- 1 piyush staff 3029 Apr 9 2018 Vagrantfile_old
Administrators-MacBook-Pro-4:aerospike-vm piyush$

Why is goapp test is looking for files in /tmp?

I'm trying to get tests working on my local dev machine Cloud SDK version is: 159.0.0
Everything I've read says that I should no change GOROOT so I'm not sure how to fix this.
$ /Users/bryan/google-cloud-sdk/platform/google_appengine/goroot/bin/goapp test
go: cannot find GOROOT directory: /tmp/go_sdk887571938/appengine/go_appengine/goroot
bryan#Bryans-MacBook Thu Jun 15 10:22:37 ~/go/src/skincarereview
$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/bryan/go/"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GO15VENDOREXPERIMENT="1"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"
bryan#Bryans-MacBook Thu Jun 15 10:22:57 ~/go/src/skincarereview
$ ls ~/google-cloud-sdk
total 408
drwxr-xr-x 30 bryan staff 1020 Jun 14 20:31 .install
-rw-r--r-- 1 bryan staff 980 Jun 14 20:30 LICENSE
-rw-r--r-- 1 bryan staff 673 Jun 14 20:30 README
-rw-r--r-- 1 bryan staff 162673 Jun 14 20:30 RELEASE_NOTES
-rw-r--r-- 1 bryan staff 8 Jun 14 20:30 VERSION
drwxr-xr-x 10 bryan staff 340 Jun 14 20:30 bin
-rw-r--r-- 1 bryan staff 2734 Jun 14 20:30 completion.bash.inc
-rw-r--r-- 1 bryan staff 2083 Jun 14 20:30 completion.zsh.inc
drwxr-xr-x 3 bryan staff 102 Jun 14 20:30 help
-rwxr-xr-x 1 bryan staff 1581 Jun 14 20:30 install.bat
-rwxr-xr-x 1 bryan staff 3471 Jun 14 20:30 install.sh
drwxr-xr-x 10 bryan staff 340 Jun 14 20:30 lib
-rw-r--r-- 1 bryan staff 308 Jun 14 20:30 path.bash.inc
-rw-r--r-- 1 bryan staff 1210 Jun 14 20:30 path.fish.inc
-rw-r--r-- 1 bryan staff 31 Jun 14 20:30 path.zsh.inc
drwxr-xr-x 6 bryan staff 204 Jun 14 20:30 platform
-rw-r--r-- 1 bryan staff 40 Jun 14 20:30 properties
bryan#Bryans-MacBook Thu Jun 15 10:24:22 ~/go/src/skincarereview
$ find / -name goroot 2>/dev/null
/Users/bryan/google-cloud-sdk/platform/google_appengine/goroot
bryan#Bryans-MacBook Thu Jun 15 10:28:43 ~/go/src/skincarereview
$ echo $PATH
/Users/bryan/google-cloud-sdk/bin:/Users/bryan/go/src/:/Users/bryan/google-cloud-sdk/platform/google_appengine/goroot/bin/:/Users/bryan/google-cloud-sdk/bin/:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin
Looking at main.go line 155, where I think the error is being thrown, I don't see where "goroot" is being set .
if fi, err := os.Stat(goroot); err != nil || !fi.IsDir() {
fmt.Fprintf(os.Stderr, "go: cannot find GOROOT directory: %v\n", goroot)
os.Exit(2)
}
You can try and follow the recommendation of this answer:
If your install is messed up beyond reason (happened to me once), just remove the cloud SDK and any references to it in your $PATH. Also completely uninstall the regular Go installation.
Then start from scratch. Install Go, unpack google-cloud-sdk, run installer (add to $PATH if needed), gcloud components install app-engine-go.
Voila.
As mentioned in that same answer, you don't set GOROOT anywhere, it is set for you.
You might want to read up on how go run/test works. The program basically builds into a temporary location and then runs, so that might be why it's complaining about your goroot.
This question could possibly help you: Golang: tests and working directory
I believe you are calling wrong goapp. You should be calling one that is directly in google_appengine folder as follows:
/Users/bryan/google-cloud-sdk/platform/google_appengine/goapp test
If you don't have goapp in google_appengine for some reason, I'll be quite surprised, but then following should work too. (Still the above is the correct way of calling it. goapp resolves appengine's goroot as a relative path from it's own location, so it's important where goapp itself is located.)
GOROOT=/Users/bryan/google-cloud-sdk/platform/google_appengine/goroot /Users/bryan/google-cloud-sdk/platform/google_appengine/goapp test

Resources