whoosh search after database migration - database

I migrated my database from sqlite3 to postgresql with the following three steps
I create a sqlite dump file with
sqlite3 app.db .dump > app_dump.sql
I create a postgresql database and initialized it with my models.py file as
python manage.py db init
where manage.py is
SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://appuser:777#localhost/app_db'
migrate = Migrate(app, db)
manager = Manager(app)
manager.add_command('db', MigrateCommand)
if __name__ == '__main__':
manager.run()
I used only the insert lines in the dump file to populate the new database.
This worked well, however I do want to have a whoosh search in my User table.
My models.py looks like this
if sys.version_info >= (3, 0):
enable_search = False
else:
enable_search = True
import flask.ext.whooshalchemy as whooshalchemy
class User(db.Model):
__searchable__ = ['username','id']
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(100), index=True)
if enable_search:
whooshalchemy.whoosh_index(app, User)
However, the search does not seem to work at all. I fear that the table has not been properly indexed? Anybody know how I can fix this?
thanks
carl
EDIT:
Solved it... have a look at
https://gist.github.com/davb5/21fbffd7a7990f5e066c

By default flask_whooshalchemy indexes a record when the sqlalchemy session commits. So if you want to index all non-indexed-data you can try my fork which is called flask_whooshalchemyplus see the manually indexing introduction.

Related

why my new table cannot be added to my database

I decide to add new Model called Project to my project:
When I run python manage.py migrate, it shows me the below error:
class Project(models.Model):
statut_juridique=[
('per', 'personne physique' ),
('sarl', 'SARL'),
('sual', 'SUARL'),
('anony', 'SA'),
]
type_du_projet = [
('ind', 'industrie'),
('agr', 'agronome'),
('ser', 'service'),
('art', 'artisanat'),
('com', 'commerce'),
]
name = models.CharField(max_length=50)
produit = ArrayField(
ArrayField(
models.CharField(max_length=20, blank=True),
size=8,
),
size=8,
)
stat_jur = models.CharField(max_length=50, choices=statut_juridique)
type_projet = models.CharField(max_length=50, choices=type_du_projet)
Nomination = models.CharField(max_length=50)
adresse = models.CharField(max_length=200)
user = models.ForeignKey(User, related_name='projet', on_delete=models.CASCADE)
def __str__(self):
return self.name
Operations to perform: Apply all migrations: admin, auth,
businesplan, contenttypes, sessions Running migrations: Applying
contenttypes.0001_initial...Traceback (most recent call last): File
"/home/abdallah/projectdjango/oasis/venv/lib/python3.8/site-packages/django/db/backends/utils.py",
line 87, in _execute
return self.cursor.execute(sql) psycopg2.errors.DuplicateTable: relation "django_content_type" already exists
And also I can't see the new table in my Database, Can you help me please!
You probably use a database that already has some tables with their migrations.
For this case you can Try using a new database, or reset your existing database to remove duplicate tables or sometimes you can troubleshot using --fake-initial as one of django-admin cmd option:
$ python manage.py migrate --fake-initial
From Django-Doc:
Allows Django to skip an app’s initial migration if all database tables with the names of all models created by all CreateModel operations in that migration already exist. This option is intended for use when first running migrations against a database that preexisted the use of migrations. This option does not, however, check for matching database schema beyond matching table names and so is only safe to use if you are confident that your existing schema matches what is recorded in your initial migration.
Source: django-admin#cmdoption-migrate-fake-initial [django-doc]

QueryException (42S02)

Illuminate \ Database \ QueryException (42S02)
SQLSTATE[42S02]: Base table or view not found: 1146 Table 'influencingquotes.posts' doesn't exist (SQL: select count(*) as aggregate from posts where quote_title = gtav hyhui)
I am not sure why this Database\QueryException is occurring :(
The Error says that the posts table is missing, maybe you forget to run:
php artisan migrate
1.If in database you couldn't find posts table
Then you need to try to find migration file for it
which for laravel 5 located in project_root_dir\database\migrations
Then via command line
php artisan migrate
2.If in database exist table, but name is wrong , for example name is post
In post model you need to specify table name
protected $table = 'post';
If it doesn't help, you need to write us
1.what you see in database, which tables or views exists there
2.What files you see in project_root_dir\database\migrations directory.

How to access and update an already existing tables of sqlite database file in another python file

I wrote the following code to create a database file and then created a table in that database and inserted values using sql queries in python.
let say this is a file named info.py
conn = sqlite3.connect('sqlite_file.db', timeout=20)
c = conn.cursor()
#Creating a new SQLite table with 1 column
c.execute('CREATE TABLE STUDENTS_ (Name CHAR, RollNo INTEGER)')
a=['Richa', 'Swapnil', 'Jahanavi', 'Shivam', 'Mehul']
b=[122, 143, 102, 186, 110]
p=0
for r in b:
c.execute("INSERT INTO STUDENTS_ VALUES (?,?);", (a[p],b[p]))
p=p+1
It runs well and gives the expected result.
Now I want to update the same table named STUDENTS_, through another code in a different python file,I tried the code below.
This is another file named info_add.py
import sqlite3
sqlite_file = 'my_first_db.sqlite' # name of the sqlite database file
STUD = 'STUDENTS_' # name of the table to be created
conn = sqlite3.connect('sqlite_file.db', timeout=20)
c = conn.cursor()
a=['Riya', 'Vipul']
b=[160, 173]
p=0
for r in b:
c.execute("INSERT INTO STUDENTS_ VALUES (?,?);", (a[p],b[p]))
p=p+1
I Get the following error:
OperationalError: database is locked
What is this error?I know i am doing wrong, please anyone help me with a right method!!Thankyou
The "database is locked" message indicates that some other connection still has an active transaction.
Python tries to be clever and automatically starts transactions for you, so you have to ensure that you end your transactions (conn.commit()) when needed.

How can I import a PostgreSQL .pgc file into a Postgres DB

I am expecting a data set to be supplied for data migration into a new system. The legacy vendor has supplied me with a .pgc file.
What is this? Is this a data file? Google tells me its an embedded SQL Program.
How can I import this to my local Postgres DB to get at the data set?
The output of command file filename.pgc is
file energyresourcingprod.pgc
energyresourcingprod.pgc: PostgreSQL custom database dump - v1.12-0
The first few lines from text editor are:
PGDMPrenergyresourcingprod9.2.49.2.4∑±00ENCODINGENCODINGSET client_encoding = 'UTF8';
false≤00
STDSTRINGS
STDSTRINGS)SET standard_conforming_strings = 'off';
false≥126214581287energyresourcingprodDATABASErCREATE DATABASE energyresourcingprod WITH TEMPLATE = template0 ENCODING = 'UTF8' LC_COLLATE = 'C' LC_CTYPE = 'C';
$DROP DATABASE energyresourcingprod;
carerixfalse26152200publicSCHEMACREATE SCHEMA public;
DROP SCHEMA public;
The file is 300Mb and the majority of it contains hashed/base64? content:
ßû+˜)™yä⁄%(»j9≤\§^¸S∏Cîó|%ëflsfi∆†p1ñºúíñ Í∆î≈3õµ=qn
Mµ¢©]Q,uÆ<*Å™ííP’ÍOõ…∫U1Eu͡ IîfiärJ¥›•$ø...
...
Many Thanks
It's just a plain PostgreSQL dump.
Use pg_restore to load it into a database.
It's weird that they used that filename, but ultimately insignificant.

PhpMyAdmin does not list a database I have access to

When I start phpMyAdmin, it does not list a database I have access too.
I know that the db exists and that I have access too, because its is the database used by one of my Joomla site.
I picked up the credential from my joomla configuration to the config.inc.php file.
Here is the file:
$cfg['Servers'][$i]['verbose'] = 'SQL9_MODULES';
$cfg['Servers'][$i]['host'] = 'sql9.modules';
$cfg['Servers'][$i]['port'] = '';
$cfg['Servers'][$i]['socket'] = '';
$cfg['Servers'][$i]['connect_type'] = 'tcp';
$cfg['Servers'][$i]['extension'] = 'mysql';
$cfg['Servers'][$i]['auth_type'] = 'config';
$cfg['Servers'][$i]['user'] = 'xxxx';
$cfg['Servers'][$i]['password'] = 'yyy';
$cfg['DefaultLang'] = 'en';
$cfg['ServerDefault'] = 1;
$cfg['UploadDir'] = '';
$cfg['SaveDir'] = '';
I tried as mentionned in other post this command :
SELECT SCHEMA_NAME AS `Database` FROM INFORMATION_SCHEMA.SCHEMATA;
The results is empty.
I have also started the "Synchronize" function from my joomla database to my joomla database and pma found all the tables !!!!
The I tried to write some selects directly on the tables but I got a
#1046 - No database selected
Last remark, this is shared hosting, so I cannot connect as root to grand me some extra privileges !!
Thanks for your help
MySQL version : 5.1.66-0+squeeze1-log - (Debian)
phpMyAdmin version : 3.5.3
Update:
SHOW GRANTS FOR CURRENT_USER
gives
GRANT USAGE ON *.* TO 'xxxx'#'%' IDENTIFIED BY PASSWORD '*****************'
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE ON `xxxx`.* TO 'xxxx'#'%'
Any missing grants ?
Got answer from my ISP support. Needed to add the following line in the configuration:
$cfg['Servers'][$i]['only_db'] = 'my-database';
Now it works.

Resources