Background
Our app uses a MySQL DB and a couple more services.
To connect our app to these servers, we have the usernames and passwords saved in a prod.config file. If we are in dev, we use a dev.config file and so on...
Recently, I have been studying good practices in the industry ( such as the https://12factor.net/ ) and the majority of them ( if not all ) specify that information like usernames and pwd's to connect to DB and other services should not be in conifg files but rather in ENV variables.
If you have no idea what the 12 factor spec is you can check this free tutorial:
https://egghead.io/lessons/egghead-summary-concepts-of-the-twelve-factor-app
Problem
Now, at first this looks fine. Many CI tools like Travis or CircleCI already force you to do this anyway. The problem here is when your smallest app uses multiple services.
In our case, for our smallest app, we would need 13 ENV variables. Variables that wouldn't be in any specific file, they would all have to be in the ENV of the machine they run on.
I fail to see how this can be seen as a good practice. I understand the main idea of not pushing your confg files with all this sensitive data, but this approach poses several issues:
When the machine reboots, you loose all you ENV variables.
If you want to avoid the previous problem, you need to run a script on machine start, that sets these said variables, which means you would have them stored in a file, defeating the whole purpose.
Where do you save these variables? They need to be somewhere else other than you flimsy head!
Questions
How would I solve the previous issues?
Why is saving private info in ENV variables seen as a good idea?
I'm going to step back a bit here and pose a question to you: why are you trying to connect to a production database from a testing environment?
The beauty of CI tools is that they allow you to spin up Docker containers to act as testing services. In your production code, it is considered best practice to keep your passwords saved in environmental variables for two main reasons:
1.) If someone ever got a hold of your code, they would have access to your database. It requires an extra level of security that is just not realistic.
2.) If someone did get a hold of your passwords, you want to be able to change them quickly. This is easier to do if your code references environmental variables instead of hard-coded strings.
When you move to a CI system, point #2 becomes moot, but point #1 becomes exceedingly important. With Travis and CircleCI, your config file is public. If you put your production password into your config file, I (or someone much more malicious) could just go scan your file and jump into your database. I've heard stories of hackers scraping public repositories for hardcoded passwords in config files. It's even easier with a tool like CircleCI.
The environmental variables you set in Travis and CircleCI should be stored at a repository level- you shouldn't need to move variables around or save them.
Environmental variables in a production system should be set-up as part of a startup script. This is highly dependent on what kind of service you're using, so I won't go into much detail here.
Related
My site is messed up and I am trying to fix it, and regardless of it I get help, it is going to take awhile likely, and it's really important that my site be live, even if it's a crappy version with just the articles and no template.
Would it not work to make a backup of the database, install Joomla fresh (the same version) and connect it to that duplicate database (then point my domain there) and then go back to working on fixing the current site that is live now? Are there any issues I should know about going in? There's a good chance the issues are related to the template or extensions (at least my understanding so far, see my other post for details on the issue) so I would think it would be faster to do this to get a working site rather than trying to turn off and on each extension, especially when I have to do it manually (and I don't know how yet) as I can't access the backend.
If this will work, do I choose the database when I install or just install empty and then change what database it connects to or do i install empty and import the tables (and how)? Still have to figure out if I can make a clone of the database and not all the files as it takes hours.
Thanks for the help, and if I should have appended this to the other post I apologize, but I figured its a separate issue.
First, ensure you have backups of both the files and the database. Then make a local copy of your site where you will work later.
The infection may lie:
in the Joomla core files, with extra content (which is usually fairly easy to spot, for example an eval of a large base64-encoded variable);
in extra files (keep in mind that even images could contain malicious code), these would be usually triggered outside of Joomla for spamming or other nefarious purposes
in the database content.
Fix:
Apply a fresh Joomla update package over your site; you will only fix n.1 above. This may restore some functionality for the first hour of survival.
Analyse the logs, and try to figure out how they got in. You need to step up security as obviously what you have is not enough.
Install a fresh Joomla, add all extensions that your site uses, copy the images folder, then connect it to a copy of the compromised database. This will fix n.1 and 2 above (as you got rid of any extra files). This may survive until they figure out you fixed it; but if you haven't patched your security, they will hack into your site again. Keep a copy of this, and restore as needed as you proceed with the following step.
Export the db to sql format (mysqldump or phpmyadmin may come in handy), then search for any xss traces, php code, javascripts that may have been injected. Since a complete control could take days, and assuming the malicious code links elsewhere, look for strings such as "https://" and "http://"; escape / as \/ and \\\/ to account for json-encoded data as well.
Once the db is clean, your local copy is reasonably safe; update all extensions and Joomla, and use it to restore the website until you fix your security.
It might work, i mean cloning the DB as far as joomla version is the same. It won't break like that, but may fail if files for extensions are not found. This is somewhat wrong, the question is how many extensions you are using and how much cleansing you need.
On the other side you mention that the site should be 'live'. Just do everything on localhost, test, fix templates, etc. Then if you're sure you're done, use akeeba backup and deploy new version to your server without long delays.
Any kind of cleansing needs some start.
You can clean the site while live, depends on complexity.
Clean might be done offline and deployed.
Sometimes import/export custom routines are needed, so you have to make own tools for everything. It occurs with large data, like when people used to made mess inside images folder or something like that.
4 ...
It's pointless to make copies of DB. You install the same version of Joomla on your local server, then you install the same template, you copy styles etc.
Then you import data with your own tools or paid ones. Estimated time is from few hours to few days, it's just data :)
I've just finished setting a dev environment where every developers /feature/, /bugfix/ and /hotfix/* git branches are automatically built and deployed to a freshly provisioned Windows Container which hosts the webapp and services creating a test environment, for each branch to be validated before merging into master.
While this is working quite nicely, I've still only got 1 dev db per developer which is used by all of their branches.
In an ideal world, I would like each of these test containers to use their own isolated db instance, however the db is currently at about 50gb at the smallest i can get it without going and tearing out historical data which is sometime useful.
What I would really like to do, is create a docker like image for this db and then spawn a new "container" from this image which only keeps track of the diff between it's changes an the original without ever altering the original db.
Is something like this even possible or does anyone have any ideas how I might achieve this db isolation, per container without having to create a full 50gb db for each?
Ok, So after much flailing around in the dark, I think I've finally come up with a solution. #ErikEJ, thanks, you started me off in the right direction. After looking into DB Snapshots on MSSQL, I found the only way to do writable snapshots seemed to be using vss and actually creating writable disk snapshots. This too led me down a long path of at first trying locally and failing, then trying to implement iscsi and still getting nowhere. Then I stumbled upon hyper-v snapshots and had a look at what was happening under the hood there and finally came across creating differencing VHD's. So basically my solution is as below.
Create VHD with a sanitised copy of my production DB mdf and ldf file in.
Then I create differencing vhds for each environment I need, mount the differencing vhds each in their own folder eg c:\db\Issue-1234 and create a new db DB_ISSUE-1234 by atatching to the files in these folders. Only the diffs are then stored and instead of having several 50-60gig copies of the db. I only have 1 and then the differencing vhds only store the difference.
I've just got this working with 2 or 3, so not sure how robust it is and how fast these differencing vhds are going to grow, but looking very promising so far and is allowing me to spin up multiple environments for testing purposes, extremely quickly (in fact all automated by scripts in deployment).
Hope this possibly helps someone else save some time one day and please let me know if anyone has figured out a more efficient/quicker/better way to do this :)
I've read a pile of other related questions... nothing really seems to answer the question I have.
My application will integrate with several different third party sites. (ebay, paypal, google, amazon...) It is a product management system and it pushes products all over the place...
Of course since it interacts with all these sites, it needs usernames, passwords, tokens.. ect.. Now I don't think it's really a good idea to store these things raw, but I still need to be able to get them raw, so I can embed them in the XML I send, or the HTTP header.
Does anyone have a suggestion on how to store the info? Is there a rails GEM?
Storing in server environment variables is the best practice for storing credentials to the DB, third-party credentials, etc. according to Twelve-Factor App methodology. How to store them depends on what you are using and how you have it setup. This promotes keeping creds out of source control, out of the database, and local to the server environment. To access an environment variable, you can use ENV, e.g.:
ENV['something']
Concerns about limitations and security:
For those storing thousands or more passwords/credentials in env vars, here are some things to help you decide whether or not to use them, in terms of feasibility and security:
If the OS user that is running the web application or service has read-only access to the Rails application root directory and subdirectories only and therefore has read access to a well-known (relative or absolute) path of a credentials/secrets file, and a developer accidentally writes a service that uses a request param as part of the pathname to a file read into a variable returned to the client, then a user of the application could potentially remotely dump all of your creds. If you put those creds into a place much less accessible by the OS user running the application in a pathname which is not easily guessable, you will reduce the risk of that exploit being used successfully to dump those creds.
You should also do what you can to make it harder to use those credentials outside of the server environment. This way, if they dumped all the credentials via an app/service exploit but cannot use those credentials outside of that environment, then they would have much less value.
The limit of how much can be stored in env variables is likely higher than you might suppose. For example, in macOS with RVM loaded which wastes a ton of environment space with bash functions, etc., I was able to get 4278 53 char length creds (e.g. bcrypt-ed):
test.sh
#!/bin/bash
set -ev
for i in `seq 1 4278`;
do
export CRED$i='...........................................'
done
ruby -e 'puts "#{ENV.size} env vars in Ruby. First cred=#{ENV["CRED1"]}"'
output:
$ time ./test.sh
for i in `seq 1 4278`;
do
export CRED$i='...........................................'
done
seq 1 4278
ruby -e 'puts "#{ENV.size} env vars in Ruby. First cred=#{ENV["CRED1"]}"'
4319 env vars in Ruby. First cred=...........................................
real 0m0.342s
user 0m0.297s
sys 0m0.019s
When I exceeded that, I got ruby: Argument list too long.
If you were to have a service in your app that could spit out any environment variable value, then you'd obviously NOT want to store creds in env vars as it would be less secure, but in my experience I've never encountered a development situation where ENV was exposed intentionally except for something like a Java administrative console that might spit out all system properties and env vars.
If you store creds in the DB, you're at more of a risk since SQL injection exploits are typically much more common. This is one reason usually only hashes of passwords are stored in the DB and not encrypted creds to other services.
If an attacker logs into the server itself and has access to the environment of the user running the web app/service or can find and read files containing the creds, you are out of luck.
OK, so I did the dumb thing and released production code (C#, VS2010) that targeted our development database (SQL Server 2008 R2). Luckily we are not using the production database yet so I didn't have the pain of trying to recover and synchronize everything...
But, I want to prevent this from happening again when it could be much more painful. My idea is to add a table I can query at startup and determine what database I am connected to by the value returned. Production would return "PROD" and dev and test would return other values, for example.
If it makes any difference, the application talks to a WCF service to access the database so I have endpoints in the config file, not actual connection strings.
Does this make sense? How have others addressed this problem?
Thanks,
Dave
The easiest way to solve this is to not have access to production accounts. Those are stored in the Machine.config file for our .net applications. In non-.net applications this is easily duplicated, by having a config file in a common location, or (dare I say) a registry entry which holds the account information.
Most of our servers are accessed through aliases too, so no one really needs to change the connection string from environment to environment. Just grab the user from the config and the server alias in the hosts file points you to the correct server. This also removes the headache from us having to update all our config files when we switch db instances (change hardware etc.)
So even with the click once deployment and the end points. You can publish the a new endpoint URI in a machine config on the end users desktop (I'm assuming this is an internal application), and then reference that in the code.
If you absolutely can't do this, as this might be a lot of work (last place I worked had 2000 call center people, so this push was a lot more difficult, but still possible). You can always have an automated build server setup which modifies the app.config file for you as a last step of building the application for you. You then ALWAYS publish the compiled code from the automated build server. Never have the change in the app.config for something like this be a manual step in the developer's process. This will always lead to problems at some point.
Now if none of this works, your final option (done this one too), which I hated, but it worked is to look up the value off of a mapped drive. Essentially, everyone in the company has a mapped drive to say R:. This is where you have your production configuration files etc. The prod account people map to one drive location with the production values, and the devs etc. map to another with the development values. I hate this option compared to the others, but it works, and it can save you in a pinch with others become tedious and difficult (due to say office politics, setting up a build server etc.).
I'm assuming your production server has a different name than your development server, so you could simply SELECT ##SERVERNAME AS ServerName.
Not sure if this answer helps you in a assumed .net environment, but within a *nix/PHP environment, this is how I handle the same situation.
OK, so I did the dumb thing and released production code
There are a times where some app behavior is environment dependent, as you eluded to. In order to provide this ability to check between development and production environments I added the following line to global /etc/profile/profile.d/custom.sh config (CentOS):
SERVICE_ENV=dev
And in code I have a wrapper method which will grab an environment variable based on name and localize it's value making it accessible to my application code. Below is a snippet demonstrating how to check the current environment and react accordingly (in PHP):
public function __call($method, $params)
{
// Reduce chatter on production envs
// Only display debug messages if override told us to
if (($method === 'debug') &&
(CoreLib_Api_Environment_Package::getValue(CoreLib_Api_Environment::VAR_LABEL_SERVICE) === CoreLib_Api_Environment::PROD) &&
(!in_array(CoreLib_Api_Log::DEBUG_ON_PROD_OVERRIDE, $params))) {
return;
}
}
Remember, you don't want to pepper your application logic with environment checks, save for a few extreme use cases as demonstrated with snippet. Rather you should be controlling access to your production databases using DNS. For example, within your development environment the following db hostname mydatabase-db would resolve to a local server instead of your actual production server. And when you push your code to the production environment, your DNS will correctly resolve the hostname, so your code should "just work" without any environment checks.
After hours of wading through textbooks and tutorials on MSBuild and app.config manipulation, I stumbled across something called SlowCheetah - XML Transforms http://visualstudiogallery.msdn.microsoft.com/69023d00-a4f9-4a34-a6cd-7e854ba318b5 that did what I needed it to do in less than hour after first stumbling across it. Definitely recommended! From the article:
This package enables you to transform your app.config or any other XML file based on the build configuration. It also adds additional tooling to help you create XML transforms.
This package is created by Sayed Ibrahim Hashimi, Chuck England and Bill Heibert, the same Hashimi who authored THE book on MSBuild. If you're looking for a simple ubiquitous way to transform your app.config, web.config or any other XML fie based on the build configuration, look no further -- this VS package will do the job.
Yeah I know I answered my own question but I already gave points to the answer that eventually pointed me to the real answer. Now I need to go back and edit the question based on my new understanding of the problem...
Dave
I' assuming yout production serveur has a different ip address. You can simply use
SELECT CONNECTIONPROPERTY('local_net_address') AS local_net_address
Assumption: live/production web app suppresses errors being shown to end-users.
Suppose your tech support team wants to see live data but through the eyes of the development-side of the application (maybe you want to see what errors are occurring, or want to see when you've got an issue fixed using an end-user's data).
Right now we've got one database serving both the dev and live boxes (not my idea - I know it's gross).
Ideas?
Edit: Best/handy tools for implementing your suggestion?
We replicate the data back to a different database. Yes, there is a delay, but it keeps people hands out of the production servers. This also allows us to "hide" information that tech support (and other people for that matter) aren't supposed to see.
In addition to replicating data down, on production, we see who's logged into the application, and if it's a member of the company, send them to the real error page versus the happy kitten playing with a ball of yarn apologizing.
Back up and restore from live to dev on a regular basis (once, twice a day). It doesn't need to be realtime (as you might be entering data from the dev side anyway, which could cause problems).
If you have PCI or HIPAA data, make sure you don't put that in your dev environment -- that might break laws.
I generally like to have a 3-tier system for web development:
Development
Testing
Live
Most of the time testing is an exact copy of the live system, except that errors are turned on, when a new version is about to be moved live it's replaced with the new version BEFORE live is, to detect upgrade issues.
Development is completely separate from live, to allow for major changes to things like the database, or changes to the production environment.
I would firstly make errors are either emailed to someone with details of how the user got there or at minimum logged so you can watch the error log while you perform similar actions to see if you get the same messages in the log.
And yes, copying the database on the dev server/site is probably your only option. You don't want any changes made by the development team to live data and you'll probably also have changes that won't work with the production database at some point.
I wouldn't recommend doing a nightly copy as a developer might be in the middle of some new feature where they have added data and then it's erased that night. I usually copy the production database(s) to dev each time a major version is released. This also allows me to do speed testing with a lot of live data. On some systems I also change everyones password to a default so I can login easily as any user.
If your configuration permits it:
a. Add a logging function (if there isn't one already) to write messages of interest to a log file.
b. Run the unix command
tail -f < logfile.txt
which will stream the growing log file to your console.
http://www.monkey.org/cgi-bin/man2html?tail
If you have Windows, you might try this:
http://tailforwin32.sourceforge.net/