I have a collection called artists, i'd like to rename it to artist_lookups. How do I do this?
With mongoid5 / mongo ruby driver 2:
# if you need to check whether foo exists
return unless Mongoid.default_client.collections.map(&:name).include?('foo')
# rename to bar
Mongoid.default_client.use(:admin).command(
renameCollection: "#{Mongoid.default_client.database.name}.foo",
to: "#{Mongoid.default_client.database.name}.bar"
)
Very simple, in mongo shell, do that:
db.artists.renameCollection( "artist_lookups" );
if you want to drop artist_lookups if it exist:
db.artists.renameCollection( "artist_lookups", true );
Some exception you can got.
10026 – Raised if the source namespace does not exist.
10027 – Raised if the target namespace exists and dropTarget is either false or unspecified.
15967 – Raised if the target namespace is an invalid collection name.
From the Mongoid Docs:
class Band
include Mongoid::Document
store_in collection: "artists", database: "music", session: "secondary"
end
Use store_in collection: "artist_lookups" in your model. This will let you store your Artist model in the artist_lookups collection.
If you want to preserve the existing data in the artists collection, and rename it, I suggest shutting down your app temporarily, renaming the collection to artist_lookups on your MongoDB server, and then restarting the app.
db.artists.renameCollection("artist_lookups")
will work for sure.
Related
I have a mongoose find method like this,
photo.find({},{
name:1,
src:1,
likes:{$literal:[]},
dislikes:{$literal:[]},
}).then(photos => ....)
what I want is, when I run the code likes and dislikes field must be an empty array for every record.
I try this way but not working.
Unsupported projection option: likes: { $literal: 1 }
Any idea to add default value for any field in find method ?
As per mongoose, document schema is constructed during its creation. So you can also edit the schema with a default value, so for every record, when it is created it will create likes, and dislikes with empty values.
You can also do this way, if you feel the schema control is not in your hands.
https://mongoosejs.com/docs/2.7.x/docs/defaults.html
photo.find({ 'name' : '1', 'likes': {$ne: []}})
I have a search engine which calls a Cakephp action and receives which model the engine should search in eg. "Projects". The variable is called $data_type;
Right now I use this to check if the model exists:
// Check if Table really exists
if(!TableRegistry::get($data_type)){
// Send error response to view
$response = [
'success' => false,
'error' => 'Data type does not exist'
];
$this->set('response', $response);
return;
}
I'm not sure I'm doing it the right or the safest way to check if a model exists, because I don't know if the TableRegistry::get() function is vulnerable to SQL injection behind the scenes.
I also found that inputing an empty string to the get() function doesn't need in a false result??? Is there a safe solution I can implement that will solve my problem?
TableRegistry::get() is not safe to use with user input
First things first. It's probably rather complicated to inject dangerous SQL via TableRegistry::get(), but not impossible, as the alias passed in the first argument will be used as the database table name in case an auto/generic-table instance is created. However the schema lookup will most likely fail before anything else, also the name will be subject to inflection, specifically underscore and lowercase inflection, so an injection attempt like
Foo; DELETE * FROM Bar;
would end up as:
foo;d_e_l_e_t_e*f_r_o_m_bar;
This would break things as it's invalid SQL, but it won't cause further harm. The bottom line however is that TableRegistry::get() cannot be regarded as safe to use with user input!
The class of the returned instance indicates a table class' existence
TableRegistry::get() looks up and instantiates possible existing table classes for the given alias, and if that fails, it will create a so called auto/generic-table, which is an instance of \Cake\ORM\Table instead of an instance of a concrete subclass thereof.
So you could check the return value against \Cake\ORM\Table to figure whether you've retrieved an instance of an actual existing table class:
$table = TableRegistry::get($data_type);
if (get_class($table) === \Cake\ORM\Table::class) {
// not an existing table class
// ...
}
Use a whitelist
That being said, unless you're working on some kind of administration tool that explicitly needs to be able to access to all tables, the proper thing do would be to use some sort of whitelisting, as having users arbitrarily look up any tables they want could be a security risk:
$whitelist = [
'Projects',
'...'
];
if (in_array($data_type, $whitelist, true) !== true) {
// not in the whitelist, access prohibited
// ...
}
Ideally you'd go even further and apply similar restrictions to the columns that can be looked up.
You may want to checkout https://github.com/FriendsOfCake/awesome-cakephp#search for some ready made search plugins.
I am using SQLAlchemy and try to manage a model "Media" which has a many-to-one relationship with a "Booking". Is it safe to call scoped_session.delete() from within a before_commit event?
def before_commit(session):
r""" Invokes the ``before_commit`` method on all items in the session.
This allows the models to perform an update-action depending on their
new data. """
for item in session.deleted:
if hasattr(item, 'before_commit'):
item.before_commit(session, 'deleted')
for item in session.dirty:
if hasattr(item, 'before_commit'):
item.before_commit(session, 'dirty')
for item in session.new:
if hasattr(item, 'before_commit'):
item.before_commit(session, 'new')
event.listen(db.session.__class__, 'before_commit', before_commit)
class Booking(db.Model):
# ...
media = db.relationship(Media, backref='booking')
def before_commit(self, session, status):
r""" Validates the booking's data. If the booking is being deleted,
all its media will be deleted with it. """
if status == 'deleted':
# Delete all the media that is associated with this booking.
for media in self.media:
session.delete(media)
a mass delete() using either session.execute("delete...") or session.query(cls).delete() should be fine, it just emits that SQL on the current connection.
As far as session.delete(obj), it looks like before_commit() is invoked before the final flush(), so in that sense you can treat it like a before_flush() event. Try it out and you should see the DELETE being emitted, and if so then you're fine.
I am creating the bulkloader.yaml automatically from my existing schema and have trouble downloading my data due the repeated=True of my KeyProperty.
class User(ndb.Model):
firstname = ndb.StringProperty()
friends = ndb.KeyProperty(kind='User', repeated=True)
The automatic created bulkloader looks like this:
- kind: User
connector: csv
connector_options:
# TODO: Add connector options here--these are specific to each connector.
property_map:
- property: __key__
external_name: key
export_transform: transform.key_id_or_name_as_string
- property: firstname
external_name: firstname
# Type: String Stats: 2 properties of this type in this kind.
- property: friends
external_name: friends
# Type: Key Stats: 2 properties of this type in this kind.
import_transform: transform.create_foreign_key('User')
export_transform: transform.key_id_or_name_as_string
This is the error message I am getting:
google.appengine.ext.bulkload.bulkloader_errors.ErrorOnTransform: Error on transform. Property: friends External Name: friends. Code: transform.key_id_or_name_as_string Details: 'list' object has no attribute 'to_path'
What can I do please?
Possible Solution:
After Tony's tip I came up with this:
- property: friends
external_name: friends
# Type: Key Stats: 2 properties of this type in this kind.
import_transform: myfriends.stringToValue(';')
export_transform: myfriends.valueToString(';')
myfriends.py
def valueToString(delimiter):
def key_list_to_string(value):
keyStringList = []
if value == '' or value is None or value == []:
return None
for val in value:
keyStringList.append(transform.key_id_or_name_as_string(val))
return delimiter.join(keyStringList)
return key_list_to_string
And this works! The encoding is in Unicode though: UTF-8. Make sure to open the file in LibreOffice as such or you would see garbled content.
The biggest challenge is import. This is what I came up with without any luck:
def stringToValue(delimiter):
def string_to_key_list(value):
keyvalueList = []
if value == '' or value is None or value == []:
return None
for val in value.split(';'):
keyvalueList.append(transform.create_foreign_key('User'))
return keyvalueList
return string_to_key_list
I get the error message:
BadValueError: Unsupported type for property friends: <type 'function'>
According to Datastore viewer, I need to create something like this:
[datastore_types.Key.from_path(u'User', u'kave#gmail.com', _app=u's~myapp1')]
Update 2:
Tony you are to be a real expert in Bulkloader. Thanks for your help. Your solution worked!
I have moved my other question to a new thread.
But one crucial problem that appears is that, when I create new users I can see my friends field shown as <missing> and it works fine.
Now when I use your solution to upload the data, I see for those users without any friend entries a <null> entry. Unfortunately this seems to break the model since friends can't be null.
Changing the model to reflect this, seems to be ignored.
friends = ndb.KeyProperty(kind='User', repeated=True, required=False)
How can I fix this please?
update:
digging further into it:
when the status <missing> is shown in the data viewer, in code it shows friends = []
However when I upload the data via csv I get a <null>, which translates to friends = [None]. I know this, because I exported the data into my local data storage and could follow it in code. Strangely enough if I empty the list del user.friends[:], it works as expected. There must be a beter way to set it while uploading via csv though...
Final Solution
This turns out to be a bug that hasn't been resolved since over one year.
In a nutshell, even though there is no value in csv, because a list is expected, gae makes a list with a None inside. This is game breaking, since retrieval of such a model ends up in an instant crash.
Adding a post_import_function, which deletes the lists with a None inside.
In my case:
def post_import(input_dict, instance, bulkload_state_copy):
if instance["friends"] is None:
del instance["friends"]
return instance
Finally everything works as expected.
When you are using repeated properties and exporting to a CSV, you should be doing some formatting to concatenate the list into a CSV understood format. Please check the example here on import/export of list of dates and hope it can help you.
EDIT : Adding suggestion for import transform from an earlier comment to this answer
For import, please try something like:
`from google.appengine.api import datastore
def stringToValue(delimiter):
def string_to_key_list(value):
keyvalueList = []
if value == '' or value is None or value == []: return None
for val in value.split(';'):
keyvalueList.append(datastore.Key.from_path('User', val))
return keyvalueList
return string_to_key_list`
if you have id instead of name , add like val = int(val)
I am trying to achieve a category model where name has unique=True,
but practically I can still add same category name with different cases.
i.e. I have a category called Food
I am still able to add food, FOOD, fOod, FOOd
Is their any philosophy behind this? or it is a work in progress.
Cause in real world if I think of Category Food, it will always be food, no matter what case it has used to mention itself.
Thank you in advance to look at this.
To answer my own question:
I have found I can have clean method on my model. So I added
class Category(models.Model):
name = models.CharField(max_length=200, unique=True)
def clean(self):
self.name = self.name.capitalize()
It is capitalising the first letter, which is then handled by the save method, which calls the validate_unique method to raise error.
You can use Postgre specific model field called Citext fields (case insensitive fields).
There are three option at the moment:
class CICharField(**options), class CIEmailField(**options) and class CITextField(**options)
Example:
from django.db import models
from django.contrib.postgres.fields import CICharField
class Category(models.Model):
name = CICharField(verbose_name="Name", max_length=255)
But don't forget to create an extension for the citext fields.
See here.
Basically, you have to add the extension class in the migration file, inside the operations array, before the first CreateModel operation.
# migration file
operations = [
CITextExtension(), # <------ here
migrations.CreateModel(
...
),
...,
]
Setting the column to case-insensitive collation should fix this. You may need to do it at the SQL level.