I've been trying to figure this question for the pass few days but can't seem to find any answer - Basically, I've the same code which works perfectly with a duplicate of the Kind's name modified and registerd in Indexes thereafter I follow the examples given by GAE to "entityname.put()" into ndb, same as what I did with my other Kind's entities.
However, this time the new Kind's datas I "put" are recorded into the table as Kind Entities, viewable under the tab in GAE App overview - 'Datastore Viewer' but are not reflected in the 'Datastore Indexes'
Due to the non-reflected data, I'm actually having trouble deleting the key due to the data not reflected in the Indexes.
I hope someone could advise me on this. Thks in advance
the devserver actually CREATES your indexes as they get queried on your dev. I think this could happen if you:
1- test using the old Kind
2- change the kind name
3- push it like that.
In this way, since you didn't test your queries with your new kind, the system will never create those indexes
Related
I was working on a project which is already advanced, specially the database, that has been filled by someone from my team, even though part of it has not been used yet. Some tables can be emptied and refilled as it's sample data, however most of them contains data which will be actually used or it's been used in the parts we're doing now.
The project started in CodeIgniter, by we've realized that Laravel can save us hours of work, so we're planning to migrate it. The point is that we didn't use the CodeIgniter's migration system, and we've seen in the Laravel documentation that only the table structure will be migrated, plus we have to create every migration.
The question here is if there's a way to both, create the migratinos files automatically, and to keep the relevant data that will be used in the application so we don't need to refill the database again (yep, there are kind of big tables). We thought on the seeders, but what we've seen is that they only store sample data...
You can use Laravel migration generator. It will simply help you to generate migrations from your
existing database. Check it out below. You will find in readme how to use it. Good luck.
https://github.com/Xethron/migrations-generator
Hope , it helps. Thanks. Cheers. -_-
I've been trying my very best not to ask any nosy question here in stackoverflow, but it has been almost one week since I got stuck in this problem and I couldn't find any solution.
I already have my working website built with CakePHP 3.2. What the website basically does is scrape Twitter for tweets containing a given search term, check if it's already in my database, and store it if it doesn't yet exist. Twitter's JSON response has this "tweet_id" property, and I've been using that value to check for whether I should ignore or append a specific tweet to my DB. While this might be okay while my database is small, I suspect it's going to slow things down considerably when my tables grow bigger. Thus my need for ElasticSearch.
My ElasticSearch server is running on my Arch Linux install, and I've configured my app to point to the said server. Also, I have my "Type" object named the same way as my "Tweets" table (I followed the documentation until the overview part http://book.cakephp.org/3.0/en/elasticsearch.html). This craps out an "Unknown method "alias" error, and following Google searches led me to creating an alternate pagination class since that was what some found to be the cause of the error (https://github.com/lorenzo/audit-stash/issues/4), which still doesn't fix things.
I'm not sure if I got this right. I installed the ElasticSearch plugin with the assumption that all I have to do is name the Types the same name as my tables, since to me the documentation "implies" that this should be done on top of the Blog Tutorial they did to "improve query performance".
TLDR, how is this supposed to work? Is my above assumption right? Do I name the Types differently and index everything myself? I'm not sure if there's just too much automagic, or I'm just poor at these sort of things. And yes, I'm new to frameworks (but not PHP, among other languages)
Thanks in advance!
I am unsure if this is a bug, by design (though I can't see the benefit), or if I am just being dimwitted.
Using Google App Engine development server, v1.9.8, I can only add a datastore entity if another of the same kind already exists. Otherwise the drop-down physically does not show the entity kind.
Further, even on an entity kind that does exist, I can only change attributes (or create them if it's new) if that attribute is not null on at least one of the other existing entities of that kind.
This is annoying, since in order to test things I have to first do:
entityKind(every=Possible, attribute=Set, to=Something, even=If,
i=Dont, yet=Need).put()
Refresh, and then remember to delete this line before refreshing again to avoid duplicating the entry in datastore.
Note though, that if I happen to be testing with only entities that do not use every attribute, I have to keep this dummy one just in case I want the others later.
I must, surely, be missing something here? I can't believe there isn't a way to just create datastore entities without hacking around it like this?
You are talking about creating entities manually using a Dev server. Over the past 4 years of using GAE I wished once or twice that this would be possible, but I never needed this feature. You create entities in your code, and you debug your code if something does not work.
You can (and should) write unit tests to test your code. A proper unit test creates an entity, saves it, reads it, verifies that all the properties are correct, deletes it, verifies that it was deleted. There is no code to cut and paste (or comment/uncomment), and there are no test entities left after testing is completed.
Your mistake is in thinking of the web console as a part of your workflow. It is absolutely not, and not intended to be. It is a vaguely hacked-together tool that allows you to do some basic operations on your data, and nothing else.
Interact with the datastore via code, and leave the web console for simple verifications.
I've run into a situation where we sometimes have to completely wipe an index and then re-index a collection. This process of course takes a lot of time. I don't want to allow for any or at least extended downtime in Prod. Thus, I am researching into a way in Solr to create a new Collection that is a copy of an old collection, but without the data. I can re-index this new Collection with little or any service degradation. I then want to use aliases to point the new Collection to the alias that our clients are using so that they will start using the new Collection without even knowing it.
I'm currently running 4.2, but wondering if I shouldn't upgrade to 4.7 in order to support this better. Seems like 4.2 has most of the same Collection API support.
One of the first snags I'm hitting is that the Collection I am copying has a lib folder in it with customer libraries. If possible I want to push these out to the solrhome/lib folder so that they only get loaded once. My problem with this is that if I have different versions of say a custom data importer, then I will run into classloader issues.
Has anyone successfully implemented this kind of scenario and could provide some insight into the pitfalls and successes that you had and what worked for you?
More Details...
I have many different collections that are part of this Solr cloud. I don't want to effect any of the other collections if possible while making changes to the new copied collection.
I'm also having a similar kind of situation where I might to modify solr schema and need to re index the whole data. But I don't have much downtime in production. So, we came up with a solution like..
Let's say I have a SolrCloud1 (existing one), with collection1 (It has it's own structure). I have my application running in different machine. There is a load-balancer in between my SolrCloud1 and application.
Now, create a separate SolrCloud (say, SolrCloud2) with collection1. Maintain the same structure as it was previously. Now, do the re indexing part in this SolrCloud2. When it's done, make available the new SolrCloud under the load-balancer. When the new SolrCLoud2 is up, shut down the SolrCloud1.
Thus without any production down time you'll re index the data. Users won't be able to know anything about this. Hope this will help.
I've been using web2py for a few years, but just starting with deploying to GAE.
I was able to get a simple web2py application running on GAE, then made some modifications which included schema modifications. I started getting weird errors (an insert failed because a field needed to be a float, when it was explicitly set to be a float constant), and it looks like the schema modifications were not carried out in GAE -- a check of the datastore shows entities w/ the old schema (for example, my problematic float-field is missing). These migrations worked fine locally (using SQLite).
settings.migrate is set to True, and all tables have migrate set to that value.
Not sure what code I could provide that would be helpful; all define_table calls are pretty standard.
As for the question(s):
What, if anything, do I need to do to make these migrations work? If it involves removing the current Entities for my tables (so new ones, with the right schemas, can be created), how would I go about doing that? (My Datastore Admin page is completely blank.) Might this be related to the fact that trying to truncate a table on GAE fails?
The problem was solved by changing the new fields from floats to doubles; not sure where (web2py or GAE, or the interface between the two) the issue w/ floats came from, but this was an easy way to route around it.