Creating a Composite Index for AppEngine in Android-Studio-based project - google-app-engine

I used Android Studio to create both an Android project, and its backend AppEngine Endpoints counterpart. I have a datastore for which I am using Objectify. The system worked great, until I added a filter to my Query (to show only specific given emails).
Query<Report> query = ofy().load().type(Report.class).filter("email", user.getEmail()).order("email").order("-when").limit(limit);
This is the POJO Datastore Entity:
#Entity
public class Report {
#Id
Long id;
String who;
#Index
Date when;
String what;
#Index
String email;
}
However, I receive such an error from the Google API Explorer when I attempt to test it:
com.google.appengine.api.datastore.DatastoreNeedIndexException: no matching index found.
The suggested index for this query is:
<datastore-index kind=\"AccessReport\" ancestor=\"false\" source=\"manual\">
<property name=\"email\" direction=\"asc\"/>
<property name=\"when\" direction=\"desc\"/>
</datastore-index>
As I understand it, I simply need to create a composite index including the specific fields email and when, with their specific sort direction.
However, most documentation that I find tell me to edit datastore-indexes.xml.
App Engine predefines a simple index on each property of an entity. An
App Engine application can define further custom indexes in an index
configuration file named datastore-indexes.xml, which is generated in
your application's /war/WEB-INF/appengine-generated directory.
Unfortunately, this file does not seem to exist anywhere in my project.
Is anyone familiar with the way to change this when working with Android Studio?

Create datastore-indexes.xml file and put it in /WEB-INF/ folder. The content will look like this:
<datastore-indexes
autoGenerate="true">
<datastore-index kind=\"AccessReport\" ancestor=\"false\" source=\"manual\">
<property name=\"email\" direction=\"asc\"/>
<property name=\"when\" direction=\"desc\"/>
</datastore-index>
</datastore-indexes>

Related

Nifi LDAP authorization - Multiple groups in search filter through wildcards

I am trying to use a search filter for Nifi authorizer via LDAP.
The problem is that the usual search filter syntax does not seem to work.
I currently have nifi working with one of my groups, so getting LDAP to work is not the issue.
I was also able to make it work by using the complete paths to both groups, but I would like to use wildcards.
So I have my AD groups as following
prod.xxxx.be
|--PROD
|--Groups TIM application entitlements
|--(A) Nifi - Admin
|--(A) Nifi - Operator
|--(A) Nifi - User
When using the following it works, and users from both groups are populated in the users list in Nifi UI;
<property name="User Search Base">OU=PROD, DC=prod, DC=xxxx, DC=be</property>
<property name="User Object Class">person</property>
<property name="User Search Scope">SUBTREE</property>
<property name="User Search Filter">(&(memberof=CN=(A) Nifi - Admin, OU=Groups TIM Application Entitlements, OU=PROD, DC=prod, DC=xxxx, DC=be)(memberof=CN=(A) Nifi - User, OU=Groups TIM Application Entitlements, OU=PROD, DC=prod, DC=xxxx, DC=be))</property>
<property name="User Identity Attribute">CN</property>
<property name="User Group Name Attribute"></property>
<property name="User Group Name Attribute - Referenced Group Attribute"></property>
But since more groups might be added, it does not seem to be very legible.
Therefore it would be great to use wildcards.
I tried many combinations I found at https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx but none seem to work for Nifi.
Examples;
(&(objectCategory=group)(cn=(A) Nifi*))
I even tried to escape the special characters;
(&(objectCategory=group)(cn=\28A\29 Nifi*))
or
(&(objectCategory=group)(cn=\28A\29 Nifi*))
Since the config file for authorizers is an XML file, but no luck.
Any advice would be greatly appreciated.
Best regards,
Kristof
We use ambari and needed to clarify the object as character data in the template:

how to deal with datastore-indexes app engine?

for this example LINK
i try to make it but this exception occurred ??
com.google.appengine.api.datastore.DatastoreNeedIndexException: no matching index found.
The suggested index for this query is:
<datastore-index kind="Contact" ancestor="true" source="manual">
<property name="UserContacts_INTEGER_IDX" direction="asc"/>
</datastore-index>
how can i write these indexes manually ?? i try to make WEB-INF/datastore-indexes.xml
at this XML i write the following :
<?xml version="1.0" encoding="utf-8"?>
<datastore-indexes autoGenerate="true">
<datastore-index kind="Contact" ancestor="true" source="manual">
<property name="UserContacts_INTEGER_IDX" direction="asc"/>
</datastore-index>
but when Iam going to deploy this error stop me to continue deploying
An internal error occurred during: "Deploying store to Google".
XML error validating
So how can i get these indexes ??
another issues >>
when i run this code and add some properties at User class as
#Persistent (mappedBy = "userC")
public List<Contact> UserContacts =new ArrayList<Contact>();
and deploy it , the engine make indexes for the UserContacts however appear Exception due to the new property - same error above can't make indexes to them -
The best thing for you to do is run you app locally and perform some test queries on those entities.
As you have <datastore-indexes autoGenerate="true"> in your indexes config you should get a file called
WEB-INF/appengine-generated/datastore-indexes-auto.xml
in there you should get all the index definitions needed for your app to work.
Copy those index definitions to your WEB-INF/datastore-indexes.xml and update your app.
If you go to your cloud console and check on your Storage/datastore/indexes view you should be all those indexed either building or serving. Once all those indexes are on "serving" you should be good to go.

iBatis - Why is sqlMapConfig.xml unable to find the sql maps defined in it?

I have a sqlMapConfig.xml that has three SQLMaps defined in it.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE sqlMapConfig
PUBLIC "-//ibatis.apache.org//DTD SQL Map Config 2.0//EN"
"http://ibatis.apache.org/dtd/sql-map-config-2.dtd">
<sqlMapConfig>
<!-- Statement namespaces are required for Ibator -->
<settings enhancementEnabled="true" useStatementNamespaces="true"/>
<!-- Setup the transaction manager and data source that are
appropriate for your environment
-->
<transactionManager type="JDBC">
<dataSource type="SIMPLE" >
<property name="JDBC.Driver"
value="com.mysql.jdbc.Driver"/>
<property name="JDBC.ConnectionURL"
value="jdbc:mysql://localhost:3306/sug"/>
<property name="JDBC.Username"
value="root"/>
<property name="JDBC.Password"
value="admin"/>
</dataSource>
</transactionManager>
<!-- SQL Map XML files should be listed here -->
<sqlMap resource="com/tatakelabs/dbmaps/categories_SqlMap.xml" />
<sqlMap resource="com/tatakelabs/dbmaps/pro_SqlMap.xml" />
<sqlMap resource="com/tatakelabs/dbmaps/pro_category_SqlMap.xml" />
</sqlMapConfig>
I get a runtime error - Cause: java.io.IOException: Could not find resource com/tatakelabs/dbmaps/categories_SqlMap.xml
categories_SqlMap.xml is present in that location. I tried changing the location of the map xml, but that did not help. sqlMapConfig.xml validates against the DTD. categories_SqlMap.xml also validates against the right DTD. I am at my wits end trying to figure out why it can't find the resource. The sqlMap files are generated by iBator.
This was happening because the sqlmap file location was not getting copied to target. Added a copy goal and that fixed it.
I had the same problem. It appears the problem lies with the location of the config file. Thus, its in relation of the project resource structure.
I moved the config file in the same package as the mapper classes and it worked. In this case try moving all the resources to this package and update the resource attributes to:
<sqlMap resource="categories_SqlMap.xml" />
<sqlMap resource="pro_SqlMap.xml" />
<sqlMap resource="pro_category_SqlMap.xml" />
Solved it.
I moved the xml file to where the Pojo was located and provided the path as follows:
<sqlMap resource="com/heena/ibatis/model/jsr/jsr.xml" />
And it worked.
place it ...src\java\abc.xml under the Source Packages directory.
If you are using Spring, you can use a SqlMapClientFactoryBean specifying property "mappingLocations". In this property you can specify a generic path, such as "com/tatakelabs/dbmaps/*_SqlMap.xml" or with a variable such as ${mapfiles}, that will be resolved by Spring as an array of file names. This lets you omit sqlMap element in sqlMapConfig. This technique will run with iBatis 2.3.4 on. However sql-map-config-2.dtd is also contained inside iBatis.jar, so you can experience some parsing errors, as /com/ibatis/sqlmap/engine/builder/xml/sql-map-config-2.dtd may have a bug. In this case you may want to replace it inside the jar with the one from the URL:
http://ibatis.apache.org/dtd/sql-map-config-2.dtd.

Spring Security 3.1 ActiveDirectoryLdapAuthenticationProvider returning partial result exception

I am trying to authenticate users to an Active Directory Instance using spring security, I am getting an Partial Results Exception. I am going around in circles trying to figure this out. Below is my config.
security-app-context
<authentication-manager erase-credentials="true">
<authentication-provider>
<user-service>
<user name="admin#damien.com" authorities="ROLE_ADMINISTRATOR" password="123admin123" />
</user-service>
</authentication-provider>
<authentication-provider ref="ldapActiveDirectoryAuthProvider"/>
</authentication-manager>
<bean id="ldapActiveDirectoryAuthProvider"
class="org.springframework.security.ldap.authentication.ad.ActiveDirectoryLdapAuthenticationProvider">
<constructor-arg value="myDomain.com" />
<constructor-arg value="ldap://ldapurl:389/" />
<property name="convertSubErrorCodesToExceptions" value="true"/>
</bean>
Error I am getting
org.springframework.dao.IncorrectResultSizeDataAccessException: Incorrect result size: expected 1, actual 0 org.springframework.security.ldap.SpringSecurityLdapTemplate.searchForSingleEntryInternal(SpringSecurityLdapTemplate.java:239)
I am struggling to find examples and the documentation indicates I am working in the right direction.
This is from the logs
SpringSecurityLdapTemplate.java 213 - Searching for entry under DN '', base = 'dc=myDomain,dc=com', filter = '(&(objectClass=user)(userPrincipalName={0}))'
and this is what I would expect that to look like on a successful attempt from some scripts that work
Searching for entry under DN 'OU=Users and Groups,DC=one,DC=two,DC=myDomain,DC=com', base = 'OU=Users and Groups,DC=one,DC=two,DC=myDomain,DC=com', filter = '(&(objectClass=user)(userPrincipalName={0}))'
Do I need to get the DN populated? How? I have looked through the ActiveDirectoryLdapAuthenticationProvider properties and don't see a way? Also the base is off but myDomain.com is the correct domain for users e.g john.doe#myDomain.com. Has anyone come across a similar problem?
To solve this I used the default LDAP provider which enables user search base to be specified, specifying the user search base and user search filter.
<ldap-authentication-provider
user-search-base="OU=Users and Groups,DC=abc,DC=myDomain,DC=com"
user-search-filter="userPrincipalName={0}" />
A user would then be logging in with john.doe#myDomain.com but the usersearch base is more specific(abc.myDomain.com). I believe AD Spring was falling down due to this.

grails with JDO - how do I work with persistenceManager correctly

I am creating an application with google app engine and grails. I have a Controller set up for my Flex application to call. The Controller calls a service to get a list back and send it back to Flex.
The Flex client is able to get the data back one time. Also If I call the action in the browser I can call the action and get data back as well. The problem that I am finding is that it is not able to call it more than once because the app is using JDO and after the first call I get an error saying that the persistenceManager has been closed.
I have read some posts that show you how to set up a single ton and just get an instance of the persistanceManager when you need it but this does not seem to work either.
This is my first time working with JDO and I could use some advice one getting these services to work on a consistant bases.
Here is the code from the service that is actually querying the datastore.
package com.dlish.fulcrum
import com.dlish.fulcrum.PMF
import org.springframework.beans.factory.InitializingBean
import com.google.appengine.api.datastore.*
import com.dlish.fulcrum.Show
class VenueBlastService {
static transactional = true
def grailsApplication
def setting
void afterPropertiesSet()
{
this.setting = grailsApplication.config.setting
}
def persistenceManager
def getAllShows() {
def query = persistenceManager.newQuery( Show )
def showInstanceList = query.execute()
return showInstanceList
}
}
The grails app-engine plugin creates the persistenceManager object in the request scope. By default, services are singletons, which means they are created once instead of per request. Thus, if you give your service a persistenceManager instance variable, the first request will have a valid persistenceManager, but all subsequent calls will have a closed persistenceManager, because your service is still referencing the manager from the first request.
There are two ways to fix this:
1) Change the scope of your service. You can do this by placing the following in your service class:
static scope = "request"
2) pass the persistenceManager from your controller to the service when you call the service method
This is very similar to my Controller's code. Except I don't use the transactional = true, why do you want this you're only doing a read? What version of the app-engine plugin are you using?
Here's my jdoconfig.xml:
<?xml version="1.0" encoding="utf-8"?>
<persistence-manager-factory name="transactions-optional">
<property name="javax.jdo.PersistenceManagerFactoryClass"
value="org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory"/>
<property name="javax.jdo.option.ConnectionURL" value="appengine"/>
<property name="javax.jdo.option.NontransactionalRead" value="true"/>
<property name="javax.jdo.option.NontransactionalWrite" value="true"/>
<property name="javax.jdo.option.RetainValues" value="true"/>
<property name="datanucleus.appengine.autoCreateDatastoreTxns" value="true"/>
</persistence-manager-factory>

Resources