qt the best way to sync files with a sqlite db - database

Question

I'm looking for the best way to sync files in some directories with a sqlite db.
First of all I use a thread that recursively look for files filtered by extension and add they do my db.
Next I use QFileSystemWatcher to watch if files change and it's work well.
The problem is that each time I run the app I don't know if the files are changed so I need to run the thread and it take 100% of cpu of one core during the execution (about 1 minute)
So how can I do to improve this algorithm?
Thanks
Regards
A993
edit:
The code is a recursive function, similar to this function that I use to count files in a directory (also this function take 100% of cpu)
int MediaScan_Thread::recursiveCount(QDir &dir)
{
int i=dir.entryInfoList(_filters,QDir::Files).count();
foreach(QFileInfo info, dir.entryInfoList(QDir::Dirs | QDir::NoDotAndDotDot))
{
QDir subdir(info.absoluteFilePath());
i += recursiveCount(subdir);
}
return i;
}
I'm working on linux but I would develop a multiplatform app.

Answer 1

I would iterate over one entryList() list, recursing the directories and checking the file filter on the files. There isn't an easy way to recursively search for file listings over multiple threads. But once you get the file listing, parallel processing on it should be easy.
I combined the calls for count and for the file listings to one hit on the I/O because there shouldn't be any reason to do this twice. This version will keep track of a QStringList of the query so more processing can be done later.
Using foreach on a recursive function can be problematic as a copy of the list is made so I switched to using a for-loop with iterators.
The special addition is mimicing the QDir nameFilters functionality since entryList() takes a fileFilters parameter that it uses for both directories and files (not what we want).
A feature that I omitted is a recursion depth limit to avoid searching forever.
This code sample was compiled but not tested
// declare in MediaScan_Thread and set them in constructor or wherever it needs to be:
QVector<QRegExp> _nameRegExps;
QStringList _filters;
QDir::Filters _dirFilters;
// ....
void MediaScan_Thread::initFilterRegExp()
{
_nameRegExps.clear();
for (int i = 0; i < _filters.size(); ++i)
{
_nameRegExps.append( QRegExp(_filters.at(i), (_dirFilters & QDir::CaseSensitive) ? Qt::CaseSensitive : Qt::CaseInsensitive, QRegExp::Wildcard) );
}
}
int MediaScan_Thread::recursiveCountAndMatchedFiles(QDir &dir, QStringList& matchedFiles )
{
int i = 0;
QFileInfoList lst = dir.entryInfoList( QStringList() , QDir::Files | QDir::Dirs | QDir::NoDotAndDotDot );
for ( auto itr = lst.begin(); itr != lst.end(); itr++ )
{
QFileInfo &info = (*itr);
if (info.isDir())
i += recursiveCountAndMatchedFiles( info.absoluteDir(), matchedFiles );
else
{
QString fileName = info.absoluteFilePath();
for (auto iter = _nameRegExps.constBegin(), end = _nameRegExps.constEnd();
iter != end; ++iter)
{
if (iter->exactMatch(fileName)) {
i++;
matchedFiles << fileName;
break;
}
}
}
}
return i;
}