Running WSo2 EMM 1.1.0, everything has been working just fine except for one big issue.
From the moment I first click on an app in the App Management tab, the WSO2EMM_DB.h2.db file starts to steadily grow as long as the server is running, even with absolutely no changes. Eventually, it gets so big that clicking an app on that tab takes a ridiculously long time to load the list of devices using the app. We're talking 5+ minutes, it becomes completely unusable. I have checked the error logs and found no errors at all, every time.
Restarting the server does nothing to correct the issue. Even if I click an app on the App Management tab once, and never again, the database file will continue to grow. Even restarting the server and not logging into the EMM page, it will continue to grow.
The only thing I've found so far that can possibly help is keeping backup copies of the database file and overwriting the current file when it gets too big. Obviously that's not a solution, as I'd need to create a new backup file every time there's a change on the server, and eventually the database file would grow too big from that too.
It's not an issue with the H2 database either. Not only have I tried starting over fresh several times and have had the same behavior, but here is the only info I could find regarding this issue, and they were having the issue regardless of whether or not it was on H2 or MySQL.
I've been trying to find a solution for this for over a month with no success. Any help would be appreciated!
EDIT: It looks like this might be the subject of EMM-826. Unfortunately there seems to be no response to that bug report so far.
EDIT 2: EMM-826 was closed with a message saying the following:
This issue is fixed in the EMM 1.1.0 GA latest pack. Please get all the patches for the product/build the product from the latest source [ https://github.com/wso2/product-emm ] and try again.
Unfortunately, that did not work for me. I'm not sure what exactly I'm doing wrong, so I'll list the what I did to try to fix it:
Downloaded the EMM 1.1.0 zip from http://wso2.com/products/enterprise-mobility-manager/.
Downloaded the zip from https://github.com/wso2/product-emm and pasted the files from that into my EMM_HOME directory.
When that didn't work, I searched for patches and found I was only using patches 1-6. In the documentation I found I could download patches 7-12 here. Patches 9 and 10 didn't work right for some reason; causing me not to be able to reach the EMM dashboard or publisher. I could only access the Carbon manager. I was able to make patches 7, 8, 11, and 12 work though - with no change in behavior.
Here are the steps I take to reproduce the issue:
After setting a fresh copy of the EMM up, I log in to the EMM dashboard as Admin, set up a user account, and upload an app through the Publisher.
Register a device to the user account I set up. In this case, an Android device running Android 4.2.2.
From the dashboard, I go to App Management and click the app I uploaded. The list of devices loads, but from that point on, the database file starts growing and eventually, after several hours, becomes so large it the device list will never load.
Please help!
Found this happening also, from a quick look it's the WSO2EMM_DB.notifications table. Seems to keep a history of all notifications over time, and the info for app installs is taken from non-optimized queries, which degrade as the table grows. You 'could' delete all rows from the table, and it will re-populate as devices 'check back' and report their info.
But you'd probably want to write a query to just keep the latest notification of each type of each user (I'll leave that to someone else...) and as was mentioned, it is apparently fixed in the latest version.
Issue appears to be resolved in EMM 2.0, which can be found here.
Related
My site is messed up and I am trying to fix it, and regardless of it I get help, it is going to take awhile likely, and it's really important that my site be live, even if it's a crappy version with just the articles and no template.
Would it not work to make a backup of the database, install Joomla fresh (the same version) and connect it to that duplicate database (then point my domain there) and then go back to working on fixing the current site that is live now? Are there any issues I should know about going in? There's a good chance the issues are related to the template or extensions (at least my understanding so far, see my other post for details on the issue) so I would think it would be faster to do this to get a working site rather than trying to turn off and on each extension, especially when I have to do it manually (and I don't know how yet) as I can't access the backend.
If this will work, do I choose the database when I install or just install empty and then change what database it connects to or do i install empty and import the tables (and how)? Still have to figure out if I can make a clone of the database and not all the files as it takes hours.
Thanks for the help, and if I should have appended this to the other post I apologize, but I figured its a separate issue.
First, ensure you have backups of both the files and the database. Then make a local copy of your site where you will work later.
The infection may lie:
in the Joomla core files, with extra content (which is usually fairly easy to spot, for example an eval of a large base64-encoded variable);
in extra files (keep in mind that even images could contain malicious code), these would be usually triggered outside of Joomla for spamming or other nefarious purposes
in the database content.
Fix:
Apply a fresh Joomla update package over your site; you will only fix n.1 above. This may restore some functionality for the first hour of survival.
Analyse the logs, and try to figure out how they got in. You need to step up security as obviously what you have is not enough.
Install a fresh Joomla, add all extensions that your site uses, copy the images folder, then connect it to a copy of the compromised database. This will fix n.1 and 2 above (as you got rid of any extra files). This may survive until they figure out you fixed it; but if you haven't patched your security, they will hack into your site again. Keep a copy of this, and restore as needed as you proceed with the following step.
Export the db to sql format (mysqldump or phpmyadmin may come in handy), then search for any xss traces, php code, javascripts that may have been injected. Since a complete control could take days, and assuming the malicious code links elsewhere, look for strings such as "https://" and "http://"; escape / as \/ and \\\/ to account for json-encoded data as well.
Once the db is clean, your local copy is reasonably safe; update all extensions and Joomla, and use it to restore the website until you fix your security.
It might work, i mean cloning the DB as far as joomla version is the same. It won't break like that, but may fail if files for extensions are not found. This is somewhat wrong, the question is how many extensions you are using and how much cleansing you need.
On the other side you mention that the site should be 'live'. Just do everything on localhost, test, fix templates, etc. Then if you're sure you're done, use akeeba backup and deploy new version to your server without long delays.
Any kind of cleansing needs some start.
You can clean the site while live, depends on complexity.
Clean might be done offline and deployed.
Sometimes import/export custom routines are needed, so you have to make own tools for everything. It occurs with large data, like when people used to made mess inside images folder or something like that.
4 ...
It's pointless to make copies of DB. You install the same version of Joomla on your local server, then you install the same template, you copy styles etc.
Then you import data with your own tools or paid ones. Estimated time is from few hours to few days, it's just data :)
I recently upgraded both Drupal and CiviCRM to the latest versions. Drupal works fine, and so does Civi except when I move to the Civi menu, I get a message that says "Database check failed - the database looks to have been partially upgraded. You may want to reload the database with the backup and try the upgrade process again." This happened earlier and reloading the most recent backup didn't help. We had to go back quite a ways before we found one that did, then had to reload a lot of data from .CSV files and by hand. I'd rather not go through with that again.
One thing we found when comparing the development site on my WAMP desktop (which was a new install that works well) with the one on my ISP's server is that the server version contained two MyISam-format files from, or generated by, CiviCase where Civi wants to see InnoDB-format files. My ISP, far more knowlegable than I am about MySQL, converted these two files two InnoDB and the problem remains. This leaves me with two questions:
could the MyISam files be the source of the "incomplete upgrade"? and
is there some way to reset a flag that tells Civi that the database is incomplete or to run the database check manually?
Thanks for any help. Civi seems to work OK as is, but the error message will be disturbing to my end users.
That message happens when you have begun the CiviCRM database upgrade but it hasn't finished. CiviCRM edits the version number in the civicrm_domain table to flag that you're in the middle of an upgrade, and when the upgrade completes, it should remove that.
The simple way to remove the message is to go edit that in the database, but it gets set there for a reason: your database upgrade never completed.
You should restore everything to the last version where it all was working--restore both the code and the database. Play around for a bit and make sure nothing funny is happening.
Run a normal CiviCRM upgrade, replacing the files and running the upgrade script. Take note of anything that seems funny when the upgrade script runs. You might try doing a minor upgrade--just a point release--simply to be sure that any upgrade is working fine.
At this point, you should either have no problems or a much more detailed problem.
Finally, please note that there is now a CiviCRM-specific StackExchange site, which is where you'll find the most CiviCRM experts to answer your questions.
I modified a page layout in our QA environment using Apex and didn't refresh from the server (my last refresh from the server was 10 days ago). Someone else made a TON of changes using the web interface in between then. I just overwrote all of her changes!
Does anyone have any advice on how I can undo my changes and go back to hers?
Unfortunately there's no tracking of layout changes so there's no way you can just revert this — are there any other developers who may have had a later copy of the metadata and haven't refreshed theirs yet?
It can be a good idea to use ANT to run a metadata backup system so that things like this don't cause problems :) There's a guide to using the Force.com Deployment Tool here, essentially you just want to use half of that process on a regular schedule.
We have had multiple DNN sites running for quite a few months now without any issues. Twice in the last 3 days our sites have gone offline by the addition of the app_offline.htm file in the root dir.
There is only one developer with access to the sites at a coding / directory viewing level and the file is generated at weird times times when he is NOT accessing our network.
We are not publishing anything to the server ( and have not published any .net code in days ), upgrading, changing code, or even modifying content. Has anyone run into this issue?
It sounds like someone is messing with your server. Can you view the event logs to see who is accessing your server? Do you have the ability to change the passwords on the box?
Mark
Assumption: live/production web app suppresses errors being shown to end-users.
Suppose your tech support team wants to see live data but through the eyes of the development-side of the application (maybe you want to see what errors are occurring, or want to see when you've got an issue fixed using an end-user's data).
Right now we've got one database serving both the dev and live boxes (not my idea - I know it's gross).
Ideas?
Edit: Best/handy tools for implementing your suggestion?
We replicate the data back to a different database. Yes, there is a delay, but it keeps people hands out of the production servers. This also allows us to "hide" information that tech support (and other people for that matter) aren't supposed to see.
In addition to replicating data down, on production, we see who's logged into the application, and if it's a member of the company, send them to the real error page versus the happy kitten playing with a ball of yarn apologizing.
Back up and restore from live to dev on a regular basis (once, twice a day). It doesn't need to be realtime (as you might be entering data from the dev side anyway, which could cause problems).
If you have PCI or HIPAA data, make sure you don't put that in your dev environment -- that might break laws.
I generally like to have a 3-tier system for web development:
Development
Testing
Live
Most of the time testing is an exact copy of the live system, except that errors are turned on, when a new version is about to be moved live it's replaced with the new version BEFORE live is, to detect upgrade issues.
Development is completely separate from live, to allow for major changes to things like the database, or changes to the production environment.
I would firstly make errors are either emailed to someone with details of how the user got there or at minimum logged so you can watch the error log while you perform similar actions to see if you get the same messages in the log.
And yes, copying the database on the dev server/site is probably your only option. You don't want any changes made by the development team to live data and you'll probably also have changes that won't work with the production database at some point.
I wouldn't recommend doing a nightly copy as a developer might be in the middle of some new feature where they have added data and then it's erased that night. I usually copy the production database(s) to dev each time a major version is released. This also allows me to do speed testing with a lot of live data. On some systems I also change everyones password to a default so I can login easily as any user.
If your configuration permits it:
a. Add a logging function (if there isn't one already) to write messages of interest to a log file.
b. Run the unix command
tail -f < logfile.txt
which will stream the growing log file to your console.
http://www.monkey.org/cgi-bin/man2html?tail
If you have Windows, you might try this:
http://tailforwin32.sourceforge.net/