I have a pretty old DotNetNuke installation which is broken (in more than one way) and won't let me upgrade. I've set up a new installation and want to move the content. The old portal has a massive amount of data, so moving it manually will take very long.
Is there any way of migrating between portals without manually copying?
I know tools exist to do this between SharePoint portals.
I'm capable of doing SQL and even writing a few lines of code if required.
It depends on where your data is stored. Some modules will implement the "Portable" feature, which means that you can export them with content. However, there's not even a great way to export all of your content at once. You can export a portal template (from Host -> Portals), but if you're upgrading a lot of versions, the new version won't be able to handle the old version.
If your information is in a lot of different modules that don't support the "Portable" feature, you'll need to consider each of those modules separately (probably doing some sort of SQL export specific to each module).
I'm facing this issue now. PokerDIY.com has a massive amount of user-dependant data but runs like a dog. I want to start fresh on DNN 7.1 with a clean install and move the relevant data across and make new pages and content. (I guess by not moving the modules and tabs across it is a bit easier).
Considering the Users will get a new UserID I will have to use email address to map any table with a USerID from the old DB to the new one... this is going to be no small task ;)
Related
My site is messed up and I am trying to fix it, and regardless of it I get help, it is going to take awhile likely, and it's really important that my site be live, even if it's a crappy version with just the articles and no template.
Would it not work to make a backup of the database, install Joomla fresh (the same version) and connect it to that duplicate database (then point my domain there) and then go back to working on fixing the current site that is live now? Are there any issues I should know about going in? There's a good chance the issues are related to the template or extensions (at least my understanding so far, see my other post for details on the issue) so I would think it would be faster to do this to get a working site rather than trying to turn off and on each extension, especially when I have to do it manually (and I don't know how yet) as I can't access the backend.
If this will work, do I choose the database when I install or just install empty and then change what database it connects to or do i install empty and import the tables (and how)? Still have to figure out if I can make a clone of the database and not all the files as it takes hours.
Thanks for the help, and if I should have appended this to the other post I apologize, but I figured its a separate issue.
First, ensure you have backups of both the files and the database. Then make a local copy of your site where you will work later.
The infection may lie:
in the Joomla core files, with extra content (which is usually fairly easy to spot, for example an eval of a large base64-encoded variable);
in extra files (keep in mind that even images could contain malicious code), these would be usually triggered outside of Joomla for spamming or other nefarious purposes
in the database content.
Fix:
Apply a fresh Joomla update package over your site; you will only fix n.1 above. This may restore some functionality for the first hour of survival.
Analyse the logs, and try to figure out how they got in. You need to step up security as obviously what you have is not enough.
Install a fresh Joomla, add all extensions that your site uses, copy the images folder, then connect it to a copy of the compromised database. This will fix n.1 and 2 above (as you got rid of any extra files). This may survive until they figure out you fixed it; but if you haven't patched your security, they will hack into your site again. Keep a copy of this, and restore as needed as you proceed with the following step.
Export the db to sql format (mysqldump or phpmyadmin may come in handy), then search for any xss traces, php code, javascripts that may have been injected. Since a complete control could take days, and assuming the malicious code links elsewhere, look for strings such as "https://" and "http://"; escape / as \/ and \\\/ to account for json-encoded data as well.
Once the db is clean, your local copy is reasonably safe; update all extensions and Joomla, and use it to restore the website until you fix your security.
It might work, i mean cloning the DB as far as joomla version is the same. It won't break like that, but may fail if files for extensions are not found. This is somewhat wrong, the question is how many extensions you are using and how much cleansing you need.
On the other side you mention that the site should be 'live'. Just do everything on localhost, test, fix templates, etc. Then if you're sure you're done, use akeeba backup and deploy new version to your server without long delays.
Any kind of cleansing needs some start.
You can clean the site while live, depends on complexity.
Clean might be done offline and deployed.
Sometimes import/export custom routines are needed, so you have to make own tools for everything. It occurs with large data, like when people used to made mess inside images folder or something like that.
4 ...
It's pointless to make copies of DB. You install the same version of Joomla on your local server, then you install the same template, you copy styles etc.
Then you import data with your own tools or paid ones. Estimated time is from few hours to few days, it's just data :)
We're designing a system for a client where they are allowing authenticated users to upload images. We've created an API to upload the files but the client only wants the latest file and delete all previous ones so that there would only ever be one.
We've looked through the docs and can't come across a way for ADAM to handle this in both 2SXC and DNN's file system.
Internally when deleting images we see API calls like the following to the internal 2SXC API, but we're wondering if this is exposed somewhere within the public API?
https://somedomain.com/api/2sxc/app/auto/data/61393528-b401-411f-a001-f423ea46700a/b7d04e2c-c565-496c-8efb-aa133cf90d33/Photo/delete?subfolder=&isFolder=false&id=189&usePortalRoot=false&appId=3
We could probably use the same endpoint above, but we'd likely run into permission issues or changes to the APIs that could be problematic.
Thank you for any advice you can give! Perhaps #iJungleBoy can provide some thoughts on this.
As a solution from a completely different direction, if you are on the later release of 2sxc (v12.8+, v13+), and comfortable programming in C#, you might consider doing this as a "cleanup" from a Dnn Scheduled Task. This can be done with a relatively easy setup. We have a Gist in place that we use as a starter. You simply put the code in the /App_Code folder then setup a normal Dnn Scheduled Task. NOTE that you can scroll down to the first comment on the Gist to see a screenshot of a complete working setup.
Accuraty's AccuTasks template on GitHub Gists
There are two more key things to note:
You need to install Dnn's CodeDom 3.6 because the example uses the later versions C#'s string interpolation - OR remove the few $"ASL2021 - {this.GetType().Name}, Task Scheduled Email", bits or convert to string.Format() or something.
Since your task's code is NOT running in a (2sxc) module, if needed, you'll do stuff like this: 2sxc Docs - Use 2sxc Instance or App Data from External C# Code
So, if you are comfortable writing code that "finds and deletes stuff older than NN days" - this might be the way to go.
I am in the process of writing an offline-capable smartclient that will have syncing capability back to the main backend when a connection can be made. As a side note, I considered the Microsoft Sync Framework but since I'm really only going one-way I didn't feel it would buy me enough to justify it.
The question I have is related to SQLite vs. SQLCE and ClickOnce deployments. I've dealt with SQLite before (impressive little tool) and I've dealt with ClickOnce, but never together. If I setup an installer for my app via ClickOnce, how do I ensure during upgrades the local database doesn't get wiped out? Is it possible to upgrade the database (table structure, etc. if necessary) as part of the installer? Or is it better to use SQLCE for something like this? I definitely don't want to go the route of installing SQL Express or anything as the overhead would be far too high for what I am doing.
I can't speak about SQLLite, having never deployed it, but I do have some info about SQLCE.
First, you don't have to deploy it as a prerequisite. You can just include the dll's in your project. You can check this article which explains how. This gives you finite control over what version is being used, and you don't have to deal with installing it per se.
Second, I don't recommend that you deploy the database as a data file and let ClickOnce manage it. When you change that file, ClickOnce will publish it again and put it in the data directory. Then it will take the previous one and put it in the \pre subfolder, and if you have no code to handle that, your user will lose his data. So if you open the database file to look at the table structure, you might be unpleasantly surprised to get a phone call from your user about the data being gone.
If you need to retain the data between updates, I recommend you move the database to the [LocalApplicationData] folder the first time the application runs, and reference it there. Then if you need to do any updates to the structure, you can do them programmatically and control when they happen. This article explains how to do this and why.
The other advantage to putting the data in LocalApplicationData is that if the user has a problem and has to uninstall and reinstall the application, his data is retained.
Regardless of the embedded database you choose your database file (.sqlite or .sdf) will be a part of your project so you will be able to use "Build Action" and "Copy to Output Directory" properties of that file to control what happens with the file during the install/update.
If you choose "Do not copy" it will not copy the database file and if you choose "Copy if newer" it will only copy if you have a new version of your database file.
You will need to experiment a little but by using these two properties you can have full control of how and when your database file is deployed/updated...
We are building a webapp which is shipped to several client as a debian package. Each client runs his own server. But the update and support is done by us.
We make regular releases of the product, with a clean version number. Most of the users get an automatic update (by Puppet), some others don't.
We want to keep a trace of the version of the application (in order to allow the user to check the version in an "about" section, and for our support to help the user more accurately).
We plan to store the version of the code and the version of the base in our database, and to keep the info up to date automatically.
Is that a good idea ?
The other alternative we see is a file.
EDIT : The code and database schema are updated together. ( if we update to version x.y.z , both code and database go to x.y.z )
Using a table to track every change to a schema as described in this post is a good practice that I'd definitely suggest to follow.
For the application, if it is shipped independently of the database (which is not clear to me), I'd embed a file in the package (and thus not use the database to store the version of the web application).
If not and thus if both the application and the database versions are maintained in sync, then I'd just use the information stored in the database.
As a general rule, I would have both, DB version and application version. The problem here is how "private" is the database. If the database is "private" to the application, and user never modifies the schema then your initial solution is fine. In my experience, databases which accumulate several years of data stop being private, it means that users add a table or two and access data using some reporting tool; from that point on the database is not exclusively used by the application any more.
UPDATE
One more thing to consider is users (application) not being able to connect to the DB and calling for support. For this case it would be better to have version, etc.. stored on file system.
Assuming there are no compelling reasons to go with one approach or the other, I think I'd go with keeping them in the database.
I'd put them in both places. Then when running your about function you quickly check that they are both the same, and if they aren't you can display extra information about the version mismatch. If they're the same then you will only need to display one of them.
I've generally found users can do "clever" things like revert databases back to old versions by manually copying directories around "because they can" so defensively dealing with it is always a good idea.
We currently use an SVN repository to ensure everyone's local environments are kept up-to-date. However, Drupal website development is somewhat trickier in that any custom code you write (for instance, PHP code written for a node body) is stored in the DB and the changes aren't recognized by the SVN working copy.
There are a couple of developers who are presently working on the same area of a Drupal site, but we're uncertain about how to best merge our local Drupal database changes together. Committing patches of database dumps seem clumsy at best and is most likely inefficient and error-prone for this purpose.
Any suggestions about how to approach this issue is appreciated!
Unfortunately, database deployment/update is one of Drupals weak spots. See this question & answers as well as this one for some suggestions on how to deal with it.
As for CCK, you could find some hints here.
As for php code in content, I agree with googletorp in that you should avoid doing this. However, if for some reason you absolutely have to do it, you could try to reduce the code to a simple function call. Thus you'd have the function itself in a module (and this would be tracked via SVN). But then you are only a little step from removing the need for the inline code anyways ...
If you are putting php code into your database then you are doing it wrong. Some stuff are inside the database like views and cck fields plus some settings. But if you put php code inside the node body you are creating a big code maintenance problem. You should really use the API and hooks instead. Create modules instead of ugly hacks with eval etc.
All that has been said above is true and good advice.. To answer your practical question, there are a number of recent modules that you could use to transport the changes done by the various developers.
The "Features" modules is a cure the the described issue of Drupal often providing nice features, albeit storing lots of configs and structure in the DB. This module enables you to capture a feature and output it as a pseudo-module (qualifies as a module with .info and code-files and all). Here is how it works:
Select functionality/feature to export
The module analyses the modules, files, DB content that is required to rebuild that feature elsewhere
The module creates a pseudo-module that contains the instructions in #3 and outputs everything (even SQL to rebuild the stuff in the DB) into a module package (as well as sets dependencies for other modules required)
Install the pseudo-module on your new site and enable it
The pseudo-module replicates the feature you exported rebuilding DB data and all
And you can tell your boss you did it all manually with razor focus to avoid even 1 error ;)
I hope this helps - http://drupal.org/project/features
By committing patches of database dumps, do you mean taking an entire extract of the db and committing it after each change?
How about a master copy of the database? Extract all tables, views, sps, etc... into individual files, put them into svn and do your merge edits on the individual objects?