I am having Power BI pro and working on the company Google analytics. When I am trying to fetch number of users on my website from Google analytics then its showing wrong data. And when I directly open google analytics data variation is too huge only in case of number of users.
In Google Analytics UI, try to run an unsampled report for the number of users with the given time frame. The number of users from the ordinary webpage UI is generated from sampling and varies from what you have in Power BI or BigQuery.
you are facing an issue of sampling.
So Power bi's connector for google analytics has a sampling issue and an API Calls limit. The standard limitations and quotas for Google Analytics API requests are documented in Limits and Quotas on API Requests. However, Power BI Desktop and Power BI service allow you to use the following enhanced number of queries.
Queries per day: 1,500,000 Queries per 100 seconds: 4,000
Might be better to connect Google analytics to a data warehouse such as Google BigQuery, which will help you get access to historical data that you won't be able to get with Power bi's connector. I'd suggest using a third-party connector.
Related
I'm trying to setup a web app in Google App Engine standard environment using Flask and MySQL.
I know Google App Engine is capable of scaling it's instance dynamically based on the performance needed but I'm unsure how I can do this for my MySQL servers.
For the MySQL connection I try to connect to Google Cloud SQL. I know how this works for a single instance but I would like to know whether it's possible to connect multiple instances and thereby allow the possiblity to dynamically scale my storage while using the same data.
So far I've found information how to do this on Google Computing Engine instances and Google Cloud SQL via this link but couldn't find out how this would work on Google App Engine standard environment.
Each GAE service instance will connect to your Google Cloud SQL as a separate client, using the same credentials. There's nothing to worry about in this respect, except maybe the cost of scaling/sizing your Google Cloud SQL instances if you have many GAE instances or limiting the number of GAE instances to keep the performance decent. From Best practices (the whole page is probably a useful read):
App Engine has the ability to automatically create more instances when load increases. If the number of App Engine instances increases
beyond what the Cloud SQL instance is sized to handle, your Cloud SQL
instance will not be able to meet the demand. You can avoid this issue
by limiting the maximum number of App Engine instances. For more
information, see Scaling elements.
As #dan-cornilescu correctly mentioned, Google App Engine automatically scales horizontally and handles the connections to your Cloud SQL instance without you having to care about.
But taking a look at your last comment on his answer :
Thank you, I understand that Google App Engine can scale dynamically
and let the created instances connect to one Google Cloud SQL
Instance, but I'm looking for a solution for the limited space and
performance on this Cloud SQL instance. Is there a way I can connect
Google App Engine to multiple instances and so increase the database
space on Google Cloud SQL?
I will go ahead and give you some options in order to address the limited space and performance of your Cloud SQL instance.
Regarding the limited space, when creating your Cloud SQL instance you can check the option “Enable automatic storage increases” under the “Configure machine type and storage” dropdown.
Regarding having multiple instances, remember that Cloud SQL instances scale vertically but not horizontally, so the first option I would recommend you is to switch to Google Cloud Spanner if what you are seeking is horizontal scalability. At the moment is the only database within Google Cloud products that allows to do so and is the choice to go.
You can also “fake out” the horizontal scalability by creating multiple Cloud SQL instances, which is indeed a best practice. While doing so, you only have to take into consideration that the amount of instances allowed per project is 40 and you will have to request an increase if you wish to have more.
I am building a mobile app with Google Cloud as back-end. It stores information about restaurants including their menu. I would need to search based on the menu items and geo location.
I am fairly confused about which Google Cloud solutions to use. Initially I have planned to use Google App Engine with Cloud Datastore for storing data, but later found out, it doesn't support geolocation search, and even if I use the App Engine search api, I won't get 'like' text search. About using cloud SQL, I am worried about its price. I am also thinking about using Google Compute Engine and installing required MongoDB or SQL in it and use some custom search libraries.
So my question is which of the below should be an feasible and economical solution for storing large amount of data and searching using text and geo-location
GAE with Cloud Datastore
GAE with Cloud SQL
GCE with MongoDB or SQL installed
Any other feasible solutions are also welcome
Google has a page to give high-level guidance on what storage option to choose on GCP, called Choosing a storage option.
Specifically for Cloud Datastore:
Description: A scalable, fully-managed NoSQL document database for your web and mobile applications.
Good for:
Semi-structured application data
Hierarchical data
Durable key-value
Common Workloads:
User profiles
Product catalogs
Game state
Specifically for Cloud SQL:
Description: A fully-managed MySQL database service that is built on the strength and reliability of Google’s infrastructure.
Good for:
Web-frameworks
Structured data
OLTP workloads
Common Workloads:
Websites, blogs, and content management systems (CMS)
Business Intelligence (BI) applications
ERP, CRM, and eCommerce applications
As you can see, your use case could border either option. Restaurants + Menus is very similar to product catalogs.
Cloud Datastore would require less ops work/thoughts on your side. For example, you don't need to think about what type of VM instance and memory size it should use. However, to do analytics on data rather than just serving the menus, you'd probably want to do daily dumps into BigQuery.
Cloud SQL will require a little more thought at the beginning, and at small sizes will be a little more expensive than Cloud Datastore. It does give you more flexibility on the analytics side in that it doesn't really need you to dump into BigQuery.
Running a different database on GCE is certainly an option if you want more control. The trade-off here will be in requiring much more active ops work on your behalf, such as installing, patching, tuning, etc.
I've created a social networking website with the following:
FrontEnd: Java for web site and hybrid app for mobile.
Web Service: Java web service which returns JSON object.
BackEnd: Neo4j(nosql, graph based database).
I've decided to host this application in any cloud server. After all research I've decided to go with google compute engine. I'm expecting to receive 10 million concurrent users. Since this is a social networking website, the user will upload photos, Like, comments, blogs, chat and etc. My region is going to be Asia/Pacific. Google does provide pricing calculator. Please find the link below:
Click Here for Compute Engine pricing calculator
However I do not know what configuration is required to handing these much of traffic. Please if any one of you have used google compute engine, please help me in selecting the right Servers, Persistent Disk, Load Balancing and GCE Network Bandwidth in Compute Engine Section from the above link. So that I can estimate the cost I need to spend per month.
Thanks for your help in advance.
Nobody can estimate the cost but you. We do not know how well the application was developped and how you use your resources.
Deploy your application on the Google Cloud, build a performance test plan (see tools like JMeter, LoadRunner etc) and test your different assumptions (type of VM, type of disk).
Use monitoring tools (Google Compute Engine now comes with one) to measure anything you need to know (bandwidth, CPU usage and whatnot).This will allow you to estimate the cost.
That way, you will know how much each option costs and will be able to make your own informed decision.
I would like to import data from flat files stored in Google Drive into DataStore. Then use the full-text search and other query options to analyze the data using apps-script.
The script API doc shows how we can access Google Drive data from the apps-script.
Now, is there any API in apps-script to access DataStore from the scripts?
Google provides a (beta) REST API to access your data. Steps to enable are here.
However, BigQuery is usually better for the type of analysis you describe. See:
https://developers.google.com/apps-script/advanced/bigquery
At this point in time, you should consider writing your own Web Service to get you access to the DataStore. You can then access that Web Service hosted in your App Engine application from your App Scripts. A detailed example is provided over here.
Additionally, the Cloud DataStore is now provided under the Google Cloud Platform and while it is still in preview, there is an API available to interact with the Datastore. This API is exhaustive and allows for both read and write operations. But keep in mind that it is currently under Preview.
We are currently hosted on Amazon's AWS platform, however our server makes extensive use of many of the Google services API's including:
Google Maps
Google Static Maps
Google Places
During normal usage, some of these API requests can take over 1-3 seconds and that's not exactly great for user experience. I wonder if there are any performance gains if we were to host our server on top of Google App Engine instead. An example would be getting better performance if you make use of as many AWS platforms for your system rather than spreading them out between multiple cloud services provider.
Yes the roundtrip to call the api will be faster but not the actual api. It can make a noticeable difference if you are calling many google apis on the same servlet call.