Are there any HTTP headers one can use to detect a mobile users number other than this X-header x-up-calling-line-id ?
I want to detect from all browsers as much as possible to cover all platform mobiles.
I will give you the exact same answer I gave to a very similar question as it should provide some insight into what you are trying to achieve.
The ability to get the MSISDN of the user visiting the WAP site depends on a number of things.
Firstly, the user has to be on Mobile Data. If the user is on WiFi then you will not receive this information.
Secondly, the users mobile network has to support the passing of the MSISDN in the HTTP headers.
Some mobile networks send headers on all requests. Others only send if going through a specific APN. Some only send this header to specific IP addresses/blocks. I have even come across networks that send the MSISDN as a $_GET variable. You will need to check with each network that you intend to support.
For example, a particular network in South Africa used to send MSISDNs in headers until around 6 months ago, and in order to receive the MSISDN in the headers now your server address needs to be whitelisted with them.
Also remember that headers are very easy to spoof, and shouldn't be relied on unless you are guaranteed that you are the originator of the HTTP request, such as in instances where you are using Web Views inside of Android Applications - you would need to put sufficient measures in place yourself.
With all of that in mind, here is what you should be looking for:
Look through your headers for any of the following. This is not a comprehensive list of MSISDN headers at all, they are only the ones I have come across in my adventures in mobile development.
X-MSISDN
X_MSISDN
HTTP_X_MSISDN
X-UP-CALLING-LINE-ID
X_UP_CALLING_LINE_ID
HTTP_X_UP_CALLING_LINE_ID
X_WAP_NETWORK_CLIENT_MSISDN
What I do is run through the headers looking for any matches. If I don't find any matches I run through the headers again using a country-specific MSISDN regex against the values to see if there are any potential MSISDNs in the headers on keys that I do not know about. If I find a potential match I add the key and data to a list that I can go through later in order to add to my list of known MSISDN headers.
I hope this has bought some clarity. What is most important to remember is that this is not a reliable method for getting an MSISDN.
Honestly you wouldn't want this feature it all. Suppose you visit couple of web sites and they capture your phone number from the header "x-up-calling-line-id". And these web sites sell your phone number to telemarketers. Do you want to be in that situation?
Technical reason is different. Phone number is never stored on phone (usually). SIM card is identified by IMSI number and Home Location Registry (at MSC) contains an entry that maps IMSI to phone number. So to answer your question, this is not possible.
Use apache_request_headers() in PHP to get MSISDN. but, the users mobile network has to support the passing of the MSISDN in the HTTP headers.
Related
recently I need to detect user location in an app. As you can imagine, there are many (obvious) solutions but I would like to have something like TikTok has in 2021.
After testing their method with virtual and normal devices, with VPN, with GPS mocking, simulated WIFI and Mobile data networks, and email addresses from providers stipulated in different countries, I still could not figure out how they detect my location and thus provide me content. Therefore, I consider their method superior over simple GPS or IP address-based methods. However, I wonder how they do it!
Any idea?
ByteDance (the company behind TikTok) uses your IP address and internet service provider info to get your approximate location, like a lot of other websites do (e.g. Athens, Greece). Google does this too - search anything you want and then scroll down to the bottom of the page. You can find your approximate location there.
I don't know how do they get this info, however, as I have not worked with networks and such.
I want to build a a webcrawler that goes randomly around the internet and puts broken (http statuscode 4xx) image links into a database.
So far I successfully build a scraper using the node packages request and cheerio. I understand the limitations are websites that dynamically create content, so I'm thinking to switch to puppeteer. Making this as fast as possible would be nice, but is not necessary as the server should run indefinetely.
My biggest question: Where do I start to crawl?
I want the crawler to find random webpages recursively, that likely have content and might have broken links. Can someone help to find a smart approach to this problem?
List of Domains
In general, the following services provide lists of domain names:
Alexa Top 1 Million: top-1m.csv.zip (free)
CSV file containing 1 million rows with the most visited websites according to Alexas algorithms
Verisign: Top-Level Domain Zone File Information (free IIRC)
You can ask Verisign directly via the linked page to give you their list of .com and .net domains. You have to fill out a form to request the data. If I recall correctly, the list is given free of charge for research purposes (maybe also for other reasons), but it might take several weeks until you get the approval.
whoisxmlapi.com: All Registered Domains (requires payment)
The company sells all kind of lists containing information regarding domain names, registrars, IPs, etc.
premiumdrops.com: Domain Zone lists (requires payment)
Similar to the previous one, you can get lists of different domain TLDs.
Crawling Approach
In general, I would assume that the older a website, the more likely it might be that it contains broken images (but that is already a bold assumption in itself). So, you could try to crawl older websites first if you use a list that contains the date when the domain was registered. In addition, you can speed up the crawling process by using multiple instances of puppeteer.
To give you a rough idea of the crawling speed: Let's say your server can crawl 5 websites per second (which requires 10-20 parallel browser instances assuming 2-4 seconds per page), you would need roughly two days for 1 million pages (1,000,000 / 5 / 60 / 60 / 24 = 2.3).
I don't know if that's what you're looking for, but this website renders a new random website whenever you click the New Random Website button, it might be useful if you could scrape it with puppeteer.
I recently had this question myself and was able to solve it with the help of this post. To clarify what other people have said previously, you can get lists of websites from various sources. Thomas Dondorf's suggestion to use Verisign's TLD zone file information is currently outdated, as I learned when I tried contacting them. Instead, you should look at ICANN's CZDNS. This website allows you to access TLD file information (by request) for any name, not just .com and .net, allowing you to potentially crawl more websites. In terms of crawling, as you said, Puppeteer would be a great choice.
Is there a way to get all current system DNS resolver ip-addresses in codename one?
The classes NetworkManager and ConnectionRequest don't provide these information.
In general Codename One seems not to provide any way to access a devices communication status information like "Wi-Fi Mac address", "IMEI", "Mobile network type" etc. Am I right?
Do you mean something like Socket.getHostOrIP()?
There is no standardized explicit DNS lookup API though.
Most of these things aren't available across platform or even consistently within the platform. You can use Display's getProperty with "IMEI", "UDID" & "MSISDN". But that will only have any chance of working on Android and you will probably need to add the build hint for permissions.
You can also always use native code calls as explained in the developer guide and the How Do I? video.
I would suggest that you revisit your need to get these values and evaluate a way around them. E.g most advertisers no longer require them as they aren't provided by all mobile OS's.
Is there any way to get the location of a cell phone (i.e. latitude/longitude) automatically when the user views a web page? This will primarily be used outside of the United States, so if there is some international standard that the US may or may not follow, that's what I'd need to know.
You can use their IP address to get a general idea. It's not very specific, but it works.
This place has a web service that will let you get their location with PHP or another server side scriping language.
http://www.geobytes.com/IpLocator.htm?Getlocation
There is no standard. The GSM and CDMA phone systems can easily be traced as per their design but you as a non government / network operator cannot trace a cellphone without a user's permission.
Do the words invasion of privacy come to mind?
I know that google has a service that can publish your GPS location if you wish, but this is opt in and you are able to switch it off.
As a iPhone user, I know that there is some applications that actually ask the user permission before geolocating there position. So there might be something in the iPhone SDK.
That way, it is reasonable to think that there is some way to do it.
Firefox 3.5 now supports geolocation, but it's not a very popular mobile browser.
http://en-gb.www.mozilla.com/en-GB/firefox/geolocation/
I'll probably try using Latitude with it's JSON feed.
See Yahoo's Fire Eagle service. Your user would require a Fire Eagle account.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
What is the best way to programmatically send an SMS text message?
Are there any free Web Service based SMS gateways?
I know that if I happen to know the user's carrier (Sprint, AT&T, etc), I can send an SMS by emailing an address based on phone number + carrier. I would like a good solution that does not require me to know the carrier. Barring that, is there an easy way to lookup carrier given a cell phone number?
I understand that there are some services independent of the major mobile carriers that offer a per-message fee-based service, with API. What I would like to know is how such organizations tap into the SMS networks. Do they have contracts with mobile carriers to resell their services?
Use http://www.twilio.com/
They have a REST interface to send SMS's and even to establish phone calls or receive phone calls.
You even get 30$ credits to try it out.
Def. the cheapest solution you will find.
Where I work we've been using http://www.clickatell.com for sending out SMS - it looks like its about 6 or 7 cents a message. They just take http POST requests to send out a message. I don't know if you'll be able to find any good free gateways. We used to send out emails, but found they were unreliable.
I've used clickatell in the past and found them very good also.
However, You could build your own to get messages VERY cheap. All you need is: a contract which gives loads of (or unlimited) messages; windows mobile phone; and a bit of socket programming.
Write a web service (pass the number and the message) which makes a call to a program on the mobile which sends the message.
I know of at least FTSE100 company which went this route.
We got fed up with using 'free' sms gateways, very unreliable.
Now we use an sms gateway device called OutboxSMS from Felltech Ltd. It sits on our network and hooks directly into out mobile phone provider using a wireless link from it's built-in transmitter. We needed to buy a SIM card (we got a PAYG with a huge bundle of messages), which is fitted to the OutboxSMS unit. We configured an email account for it on our mail server (MS Exchange), and configured the SMTP/POP3 account on the box.
We use OPManager, this sends alerts by email, which we direct to outboxsms, it parses the message and sends a text message to our ops guys phones when something goes wrong.
We also have some shell scripts which use sendmail to send an email to outboxsms, which again is converted to text messages.
I think this one deserves a new answer. There's a new player in town, it's called Nexmo and features highly competitive prices, even compared to Twilio.
https://www.nexmo.com/
I have been doing that with a nokia phone, connected to a linux machine. I have a cron job and a script that would check a database table for new messages and use gnokii to send messages. It works great if the number of sms you are goig to send isn't to big.
You could also get a GSM transmitter and issue AT commands that send sms's. Don't know why you would want to do it this way, but it's another option. This way you won't depend on someone else service
I don't know of any free SMS services, you usually buy bulk sms'seses and use an API to send them out.
Whitepages.com has an API that will allow developers to reverse lookup a phone number. It reports the carrier on mobile number, however a lot of the time it's some non-existent-anymore carrier like Powertel or something.
Supporting Angus, I can vouch for http://www.clickatell.com. It was used at a company I used to work at. It was a very easy solution to setup and use and worked great. You just need to anticipate how many messages you intend to send out and bulk order messages. They're pretty cheap, overall.
I have used TextMagic. They have reasonable rates and a great API and account management.
Sorry, after re-reading your question i realized this is not the answer your looking for. However this is what i did for my command line program. There's a website where if you put in the telephone number it gives you the carrier. So when i entered my number it screen scraped the website, got the carrier and if the carrier is in my list, i retrieved the email of that carrier
Most companies offer a SMS-to-email kinda thing. For example myphonenumber#verizon.com or something (there's a whole list on wikipedia).
I used that to create my self a little command line application in c# that sends out text messages. However, you don't really get a "reply" and the number is a pre-assigned one from the company.
I think if you want to go the free route, this is your best bet.
Here's the wikipedia link:
SMS gateway