I was trying out google Cloud Platform for my project and was performing trial operations on app engine with data store as database.I have never used or uploaded any huge data to database neither I have done any large queries today but when I checked out the quota usage its showing high usage,operation,trasfer statistics upto 8 GB and 6 Million operations,reads writes etc.I was shocked to see such a huge usage because all I have is 2 rows of data(15kb) and a table.
Its very disappointing for me, because once the billing is enabled and incidents like this happens it will be a big concern for me because I have have to employ other codes to track usage within our self.
I am kind of sure its not my fault because the quota statistics showing decrease usage statistics with time.I'm attaching the quota usage photo with this question.Does anyone else came across this situation ?
Related
We're using AWS Lightsail PostgreSQL Database. We've been experiencing errors with our C# application timing out when using the connection to database. As I'm trying to debug the issue, I went to look at the Metric graphs in AWS. I noticed that many of the graphs have frequent gaps in the data, labeled No data available. See image below.
This graph (and most of the other metrics) shows frequent gaps in the data. I'm trying to understand if this is normal, or could be a symptom of the problem. If I go back to 2 weeks timescale, there does not appear to be any other strange behaviors in any of the metric data. For example, I do not see a point in time in the past where the CPU or memory usage went crazy. The issue started happening about a week ago, so I was hoping the metrics would have helped explained why the connections to the PostgreSQL database are failing from C#.
🔶 So I guess my question is, are those frequent gaps of No data available normal for a AWS Lightsail Postgres Database?
Other Data about the machine:
1 GB RAM, 1 vCPU, 40 GB SSD
PostgreSQL database (12.11)
In the last two weeks (the average metrics show):
CPU utilization has never gone over 20%
Database connections have never gone over 35 (usually less than 5) (actually, usually 0)
Disk queue depth never goes over 0.2
Free storage space hovers around 36.5 GB
Network receive throughput is mostly less than 1 kB/s (with one spike to 141kB/s)
Network transmit throughput is mostly less than 11kB/s with all spikes less than 11.5kB/s
I would love to view the AWS logs, but they are a month old, and when trying to view them they are filled with checkpoint starting/complete logs. They start at one month ago and each page update only takes me 2 hours forward in time (and taking ~6 seconds to fetch the logs). This would require me to do ~360 page updates, and when trying, my auth timed out. 😢
So we never figured out the reason why, but this seems like it was a problem with the AWS LightSail DB. We ended up using a snapshot to create a new clone of the DB, and wiring the C# servers to the new DB. The latency issues we were having disappeared and the metric graphs looked normal (without the strange gaps).
I wish we were able to figure out the root of the problem. ATM, we are just hoping the problem does not return.
When in doubt, clone everything! 🙃
We are running an eCommerce venture that has around 2000 unique visitors in a day. The total data is around 6 GB as of now.
We are using SQL Server as our database and in the coming months the website may scale up to 10000 users per day.
From this link deciphered that it would be best to use M1 instance but could anyone help really clueless as to what to purchase from these options.
Note: Our budget is around 170 Dollars PM.
EDIT: The number of concurrent users we have had is around 150
I'd try to fit everything in memory. If you can't due to budget, you need to make sure the disk response times are up to par for your expected load. You application can vary widely. One visit to a homepage could generate many queries, or maybe you have application caching set up - so it's hard for anyone to just tell you. You should also get solid numbers on your peak number of concurrent users so you can plan for that. You don't mention your current environment, but you can get some numbers about CPU, Disk MB Read/Writes/s and memory used to help you get the right size.
I'd look at the xlarge m1. That gives you 15GB of memory to play with. You'll be able to cache all the data you need and have some left over for the OS and also have some room to grow. CPU probably won't be your issue, but be sure to check out your current use.
If you have some time to spend on it, I'd try setting up JMeter to do some load testing and see how many concurrent users you can max out with one of the cheaper options.
This topic may be better suited to ServerFault.
I'd suggest you to check at the reserved instances in heavy category where you can get interesting discounts if you plan to run it for a year or similar.
But with that budget you should be thinking about an m1.medium instance, which might be a little tight for your requirements.
We currently have our application hosted in Google App Engine. Billing is enabled to that application. This application is still in beta that we are using for testing purpose. We have a logic of serving data from the Memcache if present, if not then we get the data from the datastore and update the memcache and serve the data. We are encountering strange behaviour related to Memcache. The data related to some keys in Memcache is getting dropped after few minutes after being set. We tried setting expiration time for the keys in the memcache, even that does not seem to work. Since the data is getting dropped from the memcache the data is again from the datastore which is increasing the billing for our application.
Currently nearly 80% of the billing is related to datastore read. The datastore read is high as the memcache is not working efficiently as it should be. Any insight why we are facing this issue would be really helpful.
Just an FYI, we are having around 75000 keys in the memcache with total size of 100 MB data. Our structure demands keeping such large number of keys in memcache, which I think should not be an issue.
Our application is being by 10 users and the billing amount per day is coming to around $40.
Thanks,
Krish
Unfortunately memcache will evict keys as and when it requires. Setting the time to expire only means the item will be in memcache for up to the expiry time.
Take a look at the docs regarding eviction.
Also, take a look at this for some more insight into ways around memcache issues.
Regarding your data structure, perhaps you could post a new question and we can see if others have advice for you.
I just enabled billing just because my daily bandwidth seemed to exceed 1 GB per day.
But I realized from the Billing History section, days before the day the billing is enabled, the daily bandwidth has exceeded 1 GB (even it ever reached 2.5 GB, and the last column says $0.18), and I was not charged anything.
How come the free version of app engine allows more than 1 GB of bandwidth? If it is so, then there is no purpose of enabling billing.
If you want your application to work reliably and serve all requests, you should enable billing when about to get over the limits. Sometimes (for example, if you are at 0.8GB and someone starts 500MB download) you may get a little bit over the quota, but this is usually rare and I would not try to build my business on this.
If you have a application that exceeds that Datastore quota of 1 Gig (Master/Slave configuration) How do you clear the condition?
I have an application called "parking-helper" and it has a Total Stored Data 100% used message.
But I've deleted 99% of the data (a few days ago) cleared the indexes,
vacuumed the indexes and cleared the memcache and also waited for at
least 2 reset cycles and yet the datastore still says 100% used.
The datastore admin shows that the datastore does not add up to more than a
meg.
I know that memcache and indexes count also. It would be nice to see the sizes of these in the Datastore Admin view (but currently we cannot see these values). But I believe I cleared them also.
How can I reset this application without the drastic measure of deleting and re-creating which means I'd have to pick another application id and re-create all the other data.
Thanks,
Ralph
It's usually just a matter of waiting. And it looks like your quota is under 100% now.