I have written my own api and I want to upload it to the server but I want to secure it so noone can access it except from my app, I have tried slim-basic-auth library but it didn't work, I don not know why...
any help with that please ?
$app->add(new Tuupola\Middleware\HttpBasicAuthentication([
"secure"=>false,
"users" => [
"userher##" => "passhere##" ]
]));
I think you will never be able to hide the api url being called by the app (an attacker with rooted android device can intercept traffic easily ), If your api response user specific data, you can add auth header in request and same verify this header at server. You can also use Cross-Origin Resource Sharing-CORS and SSL for extra layer of security.
YOUR QUESTIONS
In a glance you want to have a private API, lock your Android App to it, and solve your code issue in the API server. Let's address each one in the order I mentioned them.
PRIVATE APIs
How to protect my own api from hacking and make it private
Well I have a cruel truth to reveal to you, no such thing as a private API exists, unless you don't expose it to the Internet, aka only have it accessible inside a private network, via programs that themselves are not expose also to the internet. By other words your API can only be private if its is air gaped from the internet.
So no matter if an API doesn't have public accessible documentation or if is is protected by any kind of secret or authentication mechanisms, once is accessible from the Internet is not private any-more, and you even have tooling to help with discover them.
You can read more about it in this article section, and I extract to here some bits:
Now just because the documentation for your API is not public or doesn’t even exist, it is still discoverable by anyone having access to the applications that query your API.
Interested parties just need to set up a proxy between your application and the API to watch for all requests being made and their responses in order to build a profile of your API and understand how it works.
A proxy tool:
MiTM Proxy
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
So the lesson here is that from the moment you release a mobile app that uses your API, you can consider it to belong now to the public domain, because anyone can reverse engineer it and discover how your "private" API works, and use that to build automated attacks against it.
HOW TO LOCK AN API TO AN ANDROID APP
I have written my own api and I want to upload it to the server but I want to secure it so noone can access it except from my app
Well you bought yourself a very hard task to accomplish, that some may say that its impossible to do, but once you dig deep enough in the subject, you will be able to understand that you still have some paths to explore.
First you will need to understand the difference between WHO and WHAT is accessing your API server, followed by learning some of the most common techniques used to secure an API server, and finally you will learn that the Mobile App Attestation may be what your are looking for.
The Difference Between WHO and WHAT is Accessing the API Server
I wrote a series of articles around API and Mobile security, and in the article Why Does Your Mobile App Need An Api Key? you can read in detail the difference between WHO and WAHT is accessing your API server, but I will extract here the main take aways from it:
The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
So think about the WHO as the user that your API server will be able to Authenticate and Authorize to access the data, and think about the WHAT as the software making that request in behalf of the user.
API Security Defenses
The Basic Defenses
Now that you understand the difference between WHO vs WHAT is accessing your API server you may want to go an read my article about the basic techniques to secure an API:
In this article we will explore the most common techniques used to protect an API, including how important it is to use HTTPS to protect the communication channel between mobile app and API, how API keys are used to identify the mobile app on each API request, how user agents, captchas and IP addresses are used for bot mitigation, and finally how user authentication is important for the mobile security and api security. We will discuss each of these techniques and discuss how they impact the business risk profile, i.e. how easy they are get around.
More Advanced Defenses
You can start by read this series of articles on Mobile API Security Techniques to understand how API keys, HMAC, OAUTH and certificate pinning can be used to enhance the security and at same time how they can be abused/defeated.
Afterwards and depending on your budget and resources you may employ an array of different approaches and techniques to defend your API server, and I will start to enumerate some of the most usual ones.
You can start with reCaptcha V3, followed by Web Application Firewall(WAF) and finally if you can afford it a User Behavior Analytics(UBA) solution.
Google reCAPTCHA V3:
reCAPTCHA is a free service that protects your website from spam and abuse. reCAPTCHA uses an advanced risk analysis engine and adaptive challenges to keep automated software from engaging in abusive activities on your site. It does this while letting your valid users pass through with ease.
...helps you detect abusive traffic on your website without any user friction. It returns a score based on the interactions with your website and provides you more flexibility to take appropriate actions.
WAF - Web Application Firewall:
A web application firewall (or WAF) filters, monitors, and blocks HTTP traffic to and from a web application. A WAF is differentiated from a regular firewall in that a WAF is able to filter the content of specific web applications while regular firewalls serve as a safety gate between servers. By inspecting HTTP traffic, it can prevent attacks stemming from web application security flaws, such as SQL injection, cross-site scripting (XSS), file inclusion, and security misconfigurations.
UBA - User Behavior Analytics:
User behavior analytics (UBA) as defined by Gartner is a cybersecurity process about detection of insider threats, targeted attacks, and financial fraud. UBA solutions look at patterns of human behavior, and then apply algorithms and statistical analysis to detect meaningful anomalies from those patterns—anomalies that indicate potential threats. Instead of tracking devices or security events, UBA tracks a system's users. Big data platforms like Apache Hadoop are increasing UBA functionality by allowing them to analyze petabytes worth of data to detect insider threats and advanced persistent threats.
All this solutions work based on a negative identification model, by other words they try their best to differentiate the bad from the good by identifying what is bad, not what is good, thus they are prone to false positives, despite of the advanced technology used by some of them, like machine learning and artificial intelligence.
So you may find yourself more often than not in having to relax how you block the access to the API server in order to not affect the good users. This also means that this solutions require constant monitoring to validate that the false positives are not blocking your legit users and that at same time they are properly keeping at bay the unauthorized ones.
Regarding APIs serving mobile apps a positive identification model can be used by implementing a Mobile App Attestation solution that attests the integrity of your mobile app and device its running on before any request is made to the API server.
Mobile App attestation
Finally if you have the resources you can go even further to defend your API server and Mobile App, by building your own Mobile APP Attestation, and you can read in this article section about the overall concept of it, from where I extracted this:
The role of a Mobile App Attestation service is to authenticate what is sending the requests, thus only responding to requests coming from genuine mobile app instances and rejecting all other requests from unauthorized sources.
In order to know what is sending the requests to the API server, a Mobile App Attestation service, at run-time, will identify with high confidence that your mobile app is present, has not been tampered/repackaged, is not running in a rooted device, has not been hooked into by an instrumentation framework(Frida, xPosed, Cydia, etc.), and is not the object of a Man in the Middle Attack (MitM). This is achieved by running an SDK in the background that will communicate with a service running in the cloud to attest the integrity of the mobile app and device it is running on.
On a successful attestation of the mobile app integrity, a short time lived JWT token is issued and signed with a secret that only the API server and the Mobile App Attestation service in the cloud know. In the case that attestation fails the JWT token is signed with an incorrect secret. Since the secret used by the Mobile App Attestation service is not known by the mobile app, it is not possible to reverse engineer it at run-time even when the app has been tampered with, is running in a rooted device or communicating over a connection that is the target of a MitM attack.
The mobile app must send the JWT token in the header of every API request. This allows the API server to only serve requests when it can verify that the JWT token was signed with the shared secret and that it has not expired. All other requests will be refused. In other words a valid JWT token tells the API server that what is making the request is the genuine mobile app uploaded to the Google or Apple store, while an invalid or missing JWT token means that what is making the request is not authorized to do so, because it may be a bot, a repackaged app or an attacker making a MitM attack.
A great benefit of using a Mobile App Attestation service is its proactive and positive authentication model, which does not create false positives, and thus does not block legitimate users while it keeps the bad guys at bay.
So a Mobile App Attestation will allow your API server to identify, with a very high degree of confidence, that the request is coming from WHAT you expect, the original and unmodified APK you have uploaded to the Google Play store.
THE API SERVER CODE ISSUE
I have tried slim-basic-auth library but it didn't work, I don not know why
I am not familiar at all with the Tuupola project, but from a look into the README.md for the Slim API Skeleton, specially the section for how to get a token and then how to use it. The related code that generates the token can be found at routes/token.php, and to use the token to protect an API route you can find an example at routes/todos.php. This is all configured in the config file config/middleware.php. But I have to say that I am not impressed, security wise, with their posture, and this is because they encourage the exposure of server sensitive data via API endpoints, as we can see at routes/token.php, thus I strongly advise you to immediately delete all this endpoints if they are present in your project.
SUMMARY
In my opinion the best solution is defense in depth, by applying as many layers as you can, so that you increase the time, effort and skill-set necessary to by pass all your security layers, thus keeping at bay the script kids and occasionally hackers from abusing your API server and Mobile App.
So you should employ has much techniques as possible in both sides of the equation, mobile app and API, like the ones you have learned when reading the articles I have linked: HTTPS, API keys, User Agents, Captchas, Rate Limiting, OAuth, HMAC, Certificate Pinning, Code Obfuscation, JNI/NDK to hide secretes, WAF, UBA, etc.
In the end, the solution to use in order to protect your API server and Mobile App must be chosen in accordance with the value of what you are trying to protect and the legal requirements for that type of data, like the GDPR regulations in Europe.
GOING THE EXTRA MILE
I would strongly recommend you, to also take a look into the OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
The thing you need is to add a Variable from your client app passing to your server application. Like APP_KEY or CLIENT_ID, which allows your app connecting to the server. You can add encryption so that your server application can only decrypt it and identify the request coming from your client app.
If your app is a web application and hosted in another server, you can implement IP whitelisting in your server.
But if your app is Mobile, you need to pass like a secret_key from mobile to your server.
Is there an elegant way to prevent requests from referrers outside the application? Looking at the app.yaml documentation, it doesn't seem like this is a in-box functionality but it seems like it'd be so preferred/common that it has to be hidden somewhere rather than necessarily having to reimplement it manually for every application.
It is built in.
In app.yaml you can specify login: admin for a handler and it would accept requests just from admin or from AppEngine (e.g. cron, taskqueues, and pribably urlfetch of the app itself - the last one not 100% sure).
See docs: https://cloud.google.com/appengine/docs/python/config/appref#handlers_element
Also you can check HTTP_HEADERS like referer, IP, user agent.
Also you can issue a token and pass/check it with every request.
There are probably a few issues being conflated here.
CORS - a browser enforced security measure to prevent pages maliciously or otherwise sending data to non-origin servers. Servers cannot enforce this, only permit it. Permitting this is an application level concern (i.e. Not built in to GAE)
XSRF - a server enforced security measure to stop authenticated users having their account abused by malicious client code. This is an application level concern (i.e. Not built in to GAE)
Authentication - identifying a user or client by some set of permissions. There is some support for this with GAE (cloud endpoints, provided identity service, requiring admin login), but it typically would also be an application level concern. In the case of authorizing a client (as opposed to a user) there is no built in support.
Authorization - different sets of access based on roles/permissions. Not built in.
Other solutions:
Using Host or Origin header - trivially outfoxed by someone implementing a curl request, or any application more sophisticated
API token - fine for server to server over https but trivially compromised when used on a published client (like a web page)
Your best bet is to leverage your framework and have user accounts.
If you don't want to do this, something like XSRF (token in a header and cookie) would be enough in the general case to ensure that a web client was responding to your 'app'. This only works if the client is a web browser though, same as origin/host.
No. There is no built-in logic in GAE for that. Any support would have to exist at the level of your application-specific request routing.
I'm sending simple messages to Google Analytics from a Silverlight app. They look something like this (data changed):
http://www.google-analytics.com/collect?v=1&tid=foobar&cid=foobar&t=pageview&dp=foobar&dt=foobar
Very simple API. If I use HTTP it works beautifully.
If I use HTTPS, I run afoul of Silverlight's cross domain policies. According to the docs, google-analytics.com needs to approve the cross-domain call by hosting either a clientaccesspolicy.xml (Silverlight-specific) or crossdomain.xml (original used by FLASH). Turns out they do host crossdomain.xml, and I can see that Silverlight downloads it (via Fiddler), but apparently Silverlight doesn't like the file's content and doesn't allow HTTPS calls to that domain (System.Security.SecurityException).
So.. at least at the moment, I can't use Google Analytics from Silverlight using HTTPS. Does anyone know a work around for this issue?
Note, I can't just use HTTP, because that causes IE to issue a "Allow Mixed Content" prompt which causes problems for some of our customers. I need to use HTTPS like the rest of our app.
EDIT: OK, I took a totally different approach, using HtmlPage.Window.Invoke to call a 3 line Javascript function to do the asynchronous send rather than using WebClient in the Silverlight code. Works like a champ. Anybody see any potential problems with that?
I suspect there may be a way to make this work, because I know google-analytics.com is very cross-domain friendly for exactly this reason.
If you absolutely can't get it to work another option would be to proxy the hits. If you do this, you'll want to make sure to use the ua and uip override fields in the hit you're sending so that they show up in Google Analytics with the IP address and User Agent of the original visitor and not your server.
Here's some more info on using a proxy server with the Measurement Protocol:
https://developers.google.com/analytics/devguides/collection/protocol/v1/devguide#using-a-proxy-server
I got a problem when trying to call a remote rest webservice
https://stackoverflow.com/questions/5442175/error-on-httpwebrequest
It seems it's due to cross domain policy file missing on the remote server
http://forums.silverlight.net/forums/p/183993/419448.aspx
I don't understand the reason to have this file. It doesn't protect the user at all so for what purpose did they create this restriction ?
UPDATE: Reading the explanation from MS I still fail to see how this policy allows user to be more aware since it's all done on servers side WITHOUT EVER NOTIFYING THE USER OF ANYTHING.
UPDATE: Seems people don't agree so after all my question is not so stupid :) Answer would be Secure the service not the User ? Why on earth would twitter for example would like to protect its service without even wanting it ? If a website would want to protect itself it would just well put some login / password :) It is obvious that a site wants greatest access from browser it doesn't matter if browser uses plugin or not why would the site care ?
It's for security and making you actually OPT IN to allowing it.
MSDN explains it the best so there is no point in me re-wording it:
Using Silverlight version 4 for
cross-domain communication requires
guarding against several types of
security vulnerability that can be
used to exploit Web applications.
Cross-site forgery is a class of
exploits that becomes a threat when
allowing cross-domain calls. This
exploit involves a malicious
Silverlight control transmitting
unauthorized commands to a third-party
service, without the user's knowledge.
To prevent cross-site request forgery,
Silverlight allows only site-of-origin
communication by default for all
requests other than images and media.
For example, a Silverlight control
hosted at
http://contoso.com/mycontrol.aspx can
access only services on that same
domain by default – for example
http://contoso.com/service.svc, but
not a service at
http://fabrikam.com/service.svc. This
prevents a malicious Silverlight
control hosted on the
http://contoso.com domain from calling
unauthorized operations on a service
hosted on the http://fabrikam.com
domain.
To enable a Silverlight control to
access a service in another domain,
the service must explicitly opt-in to
allow cross-domain access. By
opting-in, a service states that the
operations it exposes can safely be
invoked by a Silverlight control,
without potentially damaging
consequences to the data that the
service stores.
Here is something to consider. Assume a world where Silverlight did not require a web service to opt into being used by a Silverlight client or a indeed a Flash client come to that.
Some malcontent manages to find a weakness in a popular web service API and writes an exploit in Silverlight or Flash. Harm is done and users of this web service are upset. Where might the finger of responsibility get pointed? What happens when it gets into the news that a Silverlight or Flash plugin was the "cause" of some breach? Whose reputation is harmed? Answer: Microsoft / Adobe even though in fact it has nothing to do with them.
Ultimately this is about public relations, about the risk of there being a perceived insecurity where none is really warranted.
By including this opt-in requirement these companies massively reduce the number of services that could possibly be exploited using their plugins. Significantly reducing the risk that their plugin would unfairly be tarnished with an uncalled for reputation. For those that do opt-in the policy files can be pointed at as the responsibility of the service provider who made an informed choice to include such a file.
Ultimately if there is a choice between risking your public perception with 100,000,000 potential users or risk some disgruntlement and bewilderment of some 100,000 potential developers I know which one I would pick.
Imagine you have a webservice, which is on your website, it displays live stocks and you want this website to be used by only your silverlight client.
Now I can write a silverlight client and put it on my website that will fetch data from your website, I can steal your data and users will feel it's my data. As there is no way o determine who is connecting it.
So silverlight loaded from my domain will not connect unless you allow from your domain. This is not for server client security but this is for cross domain security. Same as JavaScript cross domain security that you can not access any objects loaded from different domain.
We have a Silverlight app which we wrote which calls a Silverlight-enabled data service. The Silverlight app cannot require a login, as it is required to present data to the unauthenticated public.
We have some schmoe who took the time to examine our Silverlight app, one way or another figure out what service it is calling, and then wrote his own client to slurp off the data so he can post it on his site and pretend like it is his. We need to prevent this.
How can i limit my data service somehow to ONLY accept requests from my silverlight app? I tried using the allow-from domain uri setting in the clientaccesspolicy.xml file to limit access to the service only from the domain in which the silverlight app sits (say mydomain.com). This did absolutely nothing though, and the service is still serving up requests to clients from outside the domain. (I tested this by putting my SL app on a different domain under our control).
What is the proper/best/most effective way to limit the data service so only our app can use it? Thanks!!!
I'm using SL 3 and .NET 3.5.
The clientaccesspolicy.xml tells the Silverlight application which Webservice it can consume. Not preventing people accessing the Webservice.
You can try using a authentication login even though its not required. This prevents 'schmoes' accessing your webservice.
Also use Dotfuscator to prevent 'schoes' to disassemble your Silverlight application and acquire the login.
Silverlight webservice security follows the same patterns you'd use for ASP.NET security, especially services exposed to AJAX. The best way to do make use of ASP.NET's authentication.
RIA Services is an even better way to handle this. It rides on top of the ASP.NET authorization, but validates on both the client and server-side automatically to combat service spoofing. It let you take care of both client and server-side authorization by adding attributes to your methods indicating that the method requires authorized access, and by which groups or users if you need to be specific.
In addition to wire-side security and obfuscation, remember that clients can attach a debugger to Silverlight applications running in their browser. See this example from MSDN Magazine's Security IQ Test, November 2008.