I have written my own api and I want to upload it to the server but I want to secure it so noone can access it except from my app, I have tried slim-basic-auth library but it didn't work, I don not know why...
any help with that please ?
$app->add(new Tuupola\Middleware\HttpBasicAuthentication([
"secure"=>false,
"users" => [
"userher##" => "passhere##" ]
]));
I think you will never be able to hide the api url being called by the app (an attacker with rooted android device can intercept traffic easily ), If your api response user specific data, you can add auth header in request and same verify this header at server. You can also use Cross-Origin Resource Sharing-CORS and SSL for extra layer of security.
YOUR QUESTIONS
In a glance you want to have a private API, lock your Android App to it, and solve your code issue in the API server. Let's address each one in the order I mentioned them.
PRIVATE APIs
How to protect my own api from hacking and make it private
Well I have a cruel truth to reveal to you, no such thing as a private API exists, unless you don't expose it to the Internet, aka only have it accessible inside a private network, via programs that themselves are not expose also to the internet. By other words your API can only be private if its is air gaped from the internet.
So no matter if an API doesn't have public accessible documentation or if is is protected by any kind of secret or authentication mechanisms, once is accessible from the Internet is not private any-more, and you even have tooling to help with discover them.
You can read more about it in this article section, and I extract to here some bits:
Now just because the documentation for your API is not public or doesn’t even exist, it is still discoverable by anyone having access to the applications that query your API.
Interested parties just need to set up a proxy between your application and the API to watch for all requests being made and their responses in order to build a profile of your API and understand how it works.
A proxy tool:
MiTM Proxy
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
So the lesson here is that from the moment you release a mobile app that uses your API, you can consider it to belong now to the public domain, because anyone can reverse engineer it and discover how your "private" API works, and use that to build automated attacks against it.
HOW TO LOCK AN API TO AN ANDROID APP
I have written my own api and I want to upload it to the server but I want to secure it so noone can access it except from my app
Well you bought yourself a very hard task to accomplish, that some may say that its impossible to do, but once you dig deep enough in the subject, you will be able to understand that you still have some paths to explore.
First you will need to understand the difference between WHO and WHAT is accessing your API server, followed by learning some of the most common techniques used to secure an API server, and finally you will learn that the Mobile App Attestation may be what your are looking for.
The Difference Between WHO and WHAT is Accessing the API Server
I wrote a series of articles around API and Mobile security, and in the article Why Does Your Mobile App Need An Api Key? you can read in detail the difference between WHO and WAHT is accessing your API server, but I will extract here the main take aways from it:
The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
So think about the WHO as the user that your API server will be able to Authenticate and Authorize to access the data, and think about the WHAT as the software making that request in behalf of the user.
API Security Defenses
The Basic Defenses
Now that you understand the difference between WHO vs WHAT is accessing your API server you may want to go an read my article about the basic techniques to secure an API:
In this article we will explore the most common techniques used to protect an API, including how important it is to use HTTPS to protect the communication channel between mobile app and API, how API keys are used to identify the mobile app on each API request, how user agents, captchas and IP addresses are used for bot mitigation, and finally how user authentication is important for the mobile security and api security. We will discuss each of these techniques and discuss how they impact the business risk profile, i.e. how easy they are get around.
More Advanced Defenses
You can start by read this series of articles on Mobile API Security Techniques to understand how API keys, HMAC, OAUTH and certificate pinning can be used to enhance the security and at same time how they can be abused/defeated.
Afterwards and depending on your budget and resources you may employ an array of different approaches and techniques to defend your API server, and I will start to enumerate some of the most usual ones.
You can start with reCaptcha V3, followed by Web Application Firewall(WAF) and finally if you can afford it a User Behavior Analytics(UBA) solution.
Google reCAPTCHA V3:
reCAPTCHA is a free service that protects your website from spam and abuse. reCAPTCHA uses an advanced risk analysis engine and adaptive challenges to keep automated software from engaging in abusive activities on your site. It does this while letting your valid users pass through with ease.
...helps you detect abusive traffic on your website without any user friction. It returns a score based on the interactions with your website and provides you more flexibility to take appropriate actions.
WAF - Web Application Firewall:
A web application firewall (or WAF) filters, monitors, and blocks HTTP traffic to and from a web application. A WAF is differentiated from a regular firewall in that a WAF is able to filter the content of specific web applications while regular firewalls serve as a safety gate between servers. By inspecting HTTP traffic, it can prevent attacks stemming from web application security flaws, such as SQL injection, cross-site scripting (XSS), file inclusion, and security misconfigurations.
UBA - User Behavior Analytics:
User behavior analytics (UBA) as defined by Gartner is a cybersecurity process about detection of insider threats, targeted attacks, and financial fraud. UBA solutions look at patterns of human behavior, and then apply algorithms and statistical analysis to detect meaningful anomalies from those patterns—anomalies that indicate potential threats. Instead of tracking devices or security events, UBA tracks a system's users. Big data platforms like Apache Hadoop are increasing UBA functionality by allowing them to analyze petabytes worth of data to detect insider threats and advanced persistent threats.
All this solutions work based on a negative identification model, by other words they try their best to differentiate the bad from the good by identifying what is bad, not what is good, thus they are prone to false positives, despite of the advanced technology used by some of them, like machine learning and artificial intelligence.
So you may find yourself more often than not in having to relax how you block the access to the API server in order to not affect the good users. This also means that this solutions require constant monitoring to validate that the false positives are not blocking your legit users and that at same time they are properly keeping at bay the unauthorized ones.
Regarding APIs serving mobile apps a positive identification model can be used by implementing a Mobile App Attestation solution that attests the integrity of your mobile app and device its running on before any request is made to the API server.
Mobile App attestation
Finally if you have the resources you can go even further to defend your API server and Mobile App, by building your own Mobile APP Attestation, and you can read in this article section about the overall concept of it, from where I extracted this:
The role of a Mobile App Attestation service is to authenticate what is sending the requests, thus only responding to requests coming from genuine mobile app instances and rejecting all other requests from unauthorized sources.
In order to know what is sending the requests to the API server, a Mobile App Attestation service, at run-time, will identify with high confidence that your mobile app is present, has not been tampered/repackaged, is not running in a rooted device, has not been hooked into by an instrumentation framework(Frida, xPosed, Cydia, etc.), and is not the object of a Man in the Middle Attack (MitM). This is achieved by running an SDK in the background that will communicate with a service running in the cloud to attest the integrity of the mobile app and device it is running on.
On a successful attestation of the mobile app integrity, a short time lived JWT token is issued and signed with a secret that only the API server and the Mobile App Attestation service in the cloud know. In the case that attestation fails the JWT token is signed with an incorrect secret. Since the secret used by the Mobile App Attestation service is not known by the mobile app, it is not possible to reverse engineer it at run-time even when the app has been tampered with, is running in a rooted device or communicating over a connection that is the target of a MitM attack.
The mobile app must send the JWT token in the header of every API request. This allows the API server to only serve requests when it can verify that the JWT token was signed with the shared secret and that it has not expired. All other requests will be refused. In other words a valid JWT token tells the API server that what is making the request is the genuine mobile app uploaded to the Google or Apple store, while an invalid or missing JWT token means that what is making the request is not authorized to do so, because it may be a bot, a repackaged app or an attacker making a MitM attack.
A great benefit of using a Mobile App Attestation service is its proactive and positive authentication model, which does not create false positives, and thus does not block legitimate users while it keeps the bad guys at bay.
So a Mobile App Attestation will allow your API server to identify, with a very high degree of confidence, that the request is coming from WHAT you expect, the original and unmodified APK you have uploaded to the Google Play store.
THE API SERVER CODE ISSUE
I have tried slim-basic-auth library but it didn't work, I don not know why
I am not familiar at all with the Tuupola project, but from a look into the README.md for the Slim API Skeleton, specially the section for how to get a token and then how to use it. The related code that generates the token can be found at routes/token.php, and to use the token to protect an API route you can find an example at routes/todos.php. This is all configured in the config file config/middleware.php. But I have to say that I am not impressed, security wise, with their posture, and this is because they encourage the exposure of server sensitive data via API endpoints, as we can see at routes/token.php, thus I strongly advise you to immediately delete all this endpoints if they are present in your project.
SUMMARY
In my opinion the best solution is defense in depth, by applying as many layers as you can, so that you increase the time, effort and skill-set necessary to by pass all your security layers, thus keeping at bay the script kids and occasionally hackers from abusing your API server and Mobile App.
So you should employ has much techniques as possible in both sides of the equation, mobile app and API, like the ones you have learned when reading the articles I have linked: HTTPS, API keys, User Agents, Captchas, Rate Limiting, OAuth, HMAC, Certificate Pinning, Code Obfuscation, JNI/NDK to hide secretes, WAF, UBA, etc.
In the end, the solution to use in order to protect your API server and Mobile App must be chosen in accordance with the value of what you are trying to protect and the legal requirements for that type of data, like the GDPR regulations in Europe.
GOING THE EXTRA MILE
I would strongly recommend you, to also take a look into the OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
The thing you need is to add a Variable from your client app passing to your server application. Like APP_KEY or CLIENT_ID, which allows your app connecting to the server. You can add encryption so that your server application can only decrypt it and identify the request coming from your client app.
If your app is a web application and hosted in another server, you can implement IP whitelisting in your server.
But if your app is Mobile, you need to pass like a secret_key from mobile to your server.
I have an app engine service deployed in GAE (written in Node) to accept a series of click stream events from my website. The data is pushed as an CORS ajax call. I think, since the POST request can be seen in browser through the developer tools, somebody can use the app engine URL to post similar data from the browser console.( like in firefox, we can resend the URL. Chrome also has this features i guess)
A few options I see here is,
Use the firewall setting to allow only my domain to send data to the GAE.
This can still fail me since the requests can be made from the browser console repetitively)
Use a WAF ( Web Application Firewall) as a proxy to create some custom rule.
What should be my approach to secure my GAE service?
I don't think the #1 approach would actually work for you: if the post requests are visible in the browser's development tools it means they're actually made by the client, not by your website, so you can't actually use firewall rules to secure them.
Since requests are coming into the service after fundamentally originating on your own website/app (where I imagine the clicks in the mentioned sequences happen) I'd place the sanity check inside that website/app (where you already have a lot more context info available to make decisions) rather than in the service itself (where, at best, you'd have to go through loops to discover/restore the original context required to make intelligent decisions). Of course, this assumes that the website/app is not a static/dumb site and has some level of intelligence.
For example, for the case using browser development tools for replaying post requests you described - the website app could have some 'executed' flag attached to the respective request, set on the first invocation (when the external service is also triggered) and could thus simply reject any subsequent cloned/copy of the request.
With the above in place I'd then replace sending the service request from the client. Instead I'd use your website/app to create and make the service request instead, after passing through the above-mentioned sanity checks. This is almost trivial to secure with simple firewall rules - valid requests can only come in from your website/app, not from the clients. I suspect this is closer to what you had in mind when you listed the #1 approach.
I have an 3rd Party API I'm wanting to use from my Silverlight 4 Application. It's interface is simply a web request (NOT A WEB SERVICE!) with form variables. According to everything I've read, this is not possible because of security issues? What security issues? Is there no way to do this? I can understand not allowing this with web services, but with simple web requests?
I cannot use my RIA Services gateway because the data is VERY sensitive and I cannot (and do not want to) transmit it under my own SSL cert. This is RIDICULOUS!
The same security issues apply in Adobe Flash. In JavaScript, it's not possible at all.
For Silverlight, the website root must contain a clientaccesspolicy.xml permitting the call.
http://msdn.microsoft.com/en-us/library/cc197955%28v=vs.95%29.aspx
To stop Cross Site Request Forgery (XSRF). Without this measure, what's to stop an attacker from issuing requests to online banking sites (or any site) on the users behalf? Exploiting the fact he may have cookies stored which will allow the request to go through.
It's too bad the internet has turned into an assumed guilty until proven innocent situation, as I could see many useful, non-malicious, uses for sending web requests.
We are starting a very large web based service project. We are trying to decide what hosting environment to use. We would really like to use Google App Engine for scalability reasons and to eliminate the need to deal with servers ourselves.
Secure logins/registrations is very important to us, as well as using our own domain. Our target audience is not very computer savvy. For this reason, we don't want to have the users have to sign up with OpenID as this can't be done within our site. We also do not want to force our customers to sign up with Google.
As far as I can see, I am out of luck. I am hoping to have a definite answer to this question. Can I have an encrypted login to our site accessed via our domain, without having to send the customers to another site for the login (OpenID/Google).
Thanks.
The hardest part is getting around the cookie issue. While you can do secure and custom logins against https://yourdomain.appspot.com, you cannot set a cookie there that will work on http://yourdomain.com.
Here is what I propose:
When you need to log the user in, send them to https://yourdomain.appspot.com. If they enter the credentials properly, create a one-time token and place it either in the datastore or in memcache. Give it a lifetime of a few seconds.
Then redirect the user back to http://yourdomain.com/authenticate?token=mytoken (obviously substitute the names as appropriate), check to make sure that the token is valid and has not expired, and if all is clear, set the appropriate cookies and expire the token.
I think that'd work just fine. Hope it helps!
As of June 27, 2012, App Engine supports SSL for custom domains.
http://googleappengine.blogspot.com/2012/06/google-app-engine-170-released-at.html
There is nothing stopping you from creating your own authentication/registration mechanism with Google App Engine. The only problem is that Google App Engine currently only supports HTTPS via https://yourid.appspot.com and not your Google Apps Domain (i.e. https://www.foobar.com). However, this is on the product roadmap for future support (SSL for third-party domains). Note, also on the product roadmap is built-in support for OAuth & OpenID.
Update: Another option may be to use a proxy server (like Apache with mod_proxy) and map your domain to the proxy server and then the proxy server can proxy the HTTP and HTTPS requests to Google App Engine. The requests could be proxied to the appspot.com domain behind the scenes. I haven't actually done this, but I believe it should work. However, this would give you a single point of failure at the proxy server which basically defeats the purpose of Google App Engine's high-availability and scalability. This would definitely just be a short-term solution until Google supports SSL for third-party domains or OpenID.
Depending on whether your threat model can accept a non-encrypted link on the "last hop" to GAE, you can use a proxy to handle SSL from the browser. Here's a HOWTO I wrote up on using CloudFlare to get always-on SSL:
http://blorn.com/post/20185054195/ssl-for-your-domain-on-google-app-engine
This isn't structurally any different than the way SSL from Google will work, it's just that Google-provided SSL will terminate within G's network rather than just outside it. If you're trying to protect against Firesheep, CloudFlare (or any other SSL proxy) will do fine. If you're worried about snoops on the trunk connection between CF and Google, you may want a more sophisticated solution.
For our e-commerce service running on AppEngine we would like to offer the option for customers to run the stores on their custom domains (eg: www.mystore.com instead of www.enstore.com/mystore).
From a user perspective, I'd like them to enter the domain name they want to use in their preference screen and tell them how to configure their dns.
I know how you normally add domains to an AppEngine instance (through Google Apps) but I'm not sure you can automate that. And even if that's possible they would be all (hundreds) listed on our google apps page.
Anyone know if this is possible/if there is a good way to do it?
I don't think there is a way to add domains "programatically" to an AppEngine instance. Apparently, domains can only be added by using the Google Apps method that you described. This is confirmed in this SO post: How do i get foo.somedomain.com get handled by myapp.appspot.com/foo on appengine
The only options that pop to mind are the following:
HTTP Redirection
Many DNS providers support HTTP Redirection. In this case, your clients would be able to set up mystore.com and www.mystore.com to redirect to www.enstore.com/mystore. There are some obvious disadvantages with this method that might not be acceptable. First of all, with 301 and 302 redirects, the users will still be forwarded to the registered AppEngine URL: www.enstore.com/mystore, and it will show in their browser. In addition, choosing between a 301 and 302 redirect can make SEO tricky, since you'd have to get into how search engines behave with these redirects. For example most search engines will not use the original URL as a source for keywords when you use a 301 redirect.
In addition to 301 and 302 redirects, some DNS providers (like DNS Made Easy) also provide what they call a "masked hidden-iframe redirect". The page will render inside a hidden iframe, so the URL does not change in the user's browsers. However this makes SEO even more tricky, and it will not allow users to bookmark internal pages, or to reference them easily.
As you can see, this option is less than ideal, but it is one option to consider in some situations. Also note that at the moment, HTTP Redirection using 301 redirects is the suggested workaround for the Naked Domain Issue 777 on the AppEngine issue tracker.
Reverse Proxy
Another option could be to set up a small server somewhere else, like a small Amazon EC2 Instance, and set up a simple reverse proxy. You would be able to set this up very easily, just by using Apache and mod_proxy (or various other alternatives). This would allow you to ask your clients to set up a normal A Record pointing to this instance, while the Apache HTTP server would be acting as a proxy to your AppEngine.
The fundamental configuration directive to set up a reverse proxy in mod_proxy is the ProxyPass. You would typically set it up with one line like these for each VirtualHost (for each client domain):
ProxyPass / http://www.enmystore.com/mystore/
The configuration of the remote proxy could be easily handled by your back-end software.
This is a neater solution which gives you plenty of control - but there are obviously some costs for these benefits. First of all, there is the expense to host the reverse proxy. You would also be adding another point of failure, so you have to add this to your high-availability plan. In addition, if you are serving some pages through SSL it can become quite complicated.
Another option is to have each customer sign up for google apps, and then add your appengine app to their app. That way they can manage the url. They will need to use a cname for this, so urls will be limited to something like 'store.customer.com' You will have to support the multitenancy off of the host-header, but that isn't hard to do given that you already have a way to support multitenancy already. You might want to do the setup for the first couple of clients yourself so you can document the easiest way to set it up.
The rietveld code review app does this as you can add it to your google apps domain. See http://code.google.com/p/rietveld/wiki/CodeReviewHelp#Using_Code_Reviews_with_Google_Apps for more detail.
The preferred option is probably to offer your solution through the Google Solutions Marketplace: http://www.google.com/enterprise/enterprise_marketplace/about.html
We did something similar to Daniel Vassallo second proposal.
We created a python app on the Heroku cloud
(there is no limit for connecting custom domains).
This app is using python requests 1.2.0 lib to get the correct page from your app engine application according to the request domain.
all you need to tell your clients is to put your Heroku app url as their CNAME
For naked domains you can always use wwwizer