I would like to check whether
if there any ways to delete the record in the page, without deleting records in database?
I'm doing a shopping cart and would like to have the user to be able to view back their past transaction.
Is there any ways that I could delete the records in the cart, after I check out? And also able to view the past transaction.
I've not done any code yet,
just seeking for advices
Yes. You can have a database with the products, a second with your users and a third one to associate the users id with the products id they purchased.
You can never delete a item from the database. You can include a information of status like 'active/inactive' instead of actually deleting it.
Edit
When you have products in cart, usually the list is saved in cookies, temporarily by browser or javascript or another way (if your country forbid cookie, you have to search for alternatives). Once the purchase is finished, the cart list is saved in database and the cookie or whatever is cleared. You do not delete anything from database.
Example
Users/ buyers database
+--------+----------+
| idUser | nameUser |
+--------+----------+
| 1 | Antony |
| 2 | Betty |
| 3 | Carl |
+--------+----------+
Products database
+-----------+--------------+-------+-----------+
| idProduct | nameProduct | price | available |
+-----------+--------------+-------+-----------+
| 1 | Apple dozen | 10.00 | yes |
| 2 | Banana unity | 20.00 | yes |
| 3 | Cherry kg | 30.00 | yes |
+-----------+--------------+-------+-----------+
Active/ Inactive Example
Notice here how you don't need to delete any record ever. You can create a option such like available to products like example above. So you can just set it to yes or no. So you just code your front-end to show products which available is yes while your back-end can see all of them. It is very useful. In a case when your product has tens of information and you delete the product, but it will only be unavailable for 3 weeks, you would lose a great time deleting and typing and saving everything again. It is much better just set it to show or not to show. (Some stores just opt to show everything, but alert when it is unavailable).
Have in mind that it isn't a rule for everything. Always look for the best perfomance. If you have a database with 10k unavaiable products that will never be available again and you just have 2 available itens, it would be a little nosense keep all this records alive.
About prices historic
Prices Historic Database
+--------+--------+-----------+-----+----------+----------+
| idItem | idUser | idProduct | qty | subtotal | date |
+--------+--------+-----------+-----+----------+----------+
| 1 | 1 | 1 | 2 | 20 | 10/10/13 |
| 2 | 1 | 2 | 1 | 20 | 10/10/13 |
| 3 | 2 | 1 | 1 | 10 | 11/12/13 |
| 4 | 2 | 2 | 1 | 20 | 11/12/13 |
| 5 | 2 | 3 | 1 | 30 | 11/12/13 |
| 6 | 1 | 1 | 1 | 10 | 01/06/14 |
+--------+--------+-----------+-----+----------+----------+
It is always good to have a unique identification for each row on database. Well, in most cases. Its is not very readable for humans, but it fits well for machines. Check this database. The line 1 and 2 say us that user 1 bought on 10/10/13 the item with id 1 x 2 and item with id 2 x 1. Translating: Antony bought 2 Apple dozen and 1 Banana unity.
In this database you could filter rows by user and them group by Date. You would see that Antony bought more Apples in 01/06/2014.
I personally like to save the subtotal on database. If in 2015 the apple price raise to 15.00, you and the user can see he paid 10.00 in 2013 and you can do reverse calc to get the individual price.
If you could read this thid database well, you see Betty bought one of each item in 11/12/13 and Carl has never bought anything.
I believe and hope it will help you.
It is just a simplified example on how the logic works. I matched the product directly with buyer, but most stores usually add a 4th database just to list Orders. So you relate Users with Orders and Orders with Products. Everything has its advantages and disadvantages
Related
Question
main question
How can I ephemerally materialize slowly changing dimension type 2 from from a folder of daily extracts, where each csv is one full extract of a table from from a source system?
rationale
We're designing ephemeral data warehouses as data marts for end users that can be spun up and burned down without consequence. This requires we have all data in a lake/blob/bucket.
We're ripping daily full extracts because:
we couldn't reliably extract just the changeset (for reasons out of our control), and
we'd like to maintain a data lake with the "rawest" possible data.
challenge question
Is there a solution that could give me the state as of a specific date and not just the "newest" state?
existential question
Am I thinking about this completely backwards and there's a much easier way to do this?
Possible Approaches
custom dbt materialization
There's a insert_by_period dbt materialization in the dbt.utils package, that I think might be exactly what I'm looking for? But I'm confused as it's dbt snapshot, but:
run dbt snapshot for each file incrementally, all at once; and,
built directly off of an external table?
Delta Lake
I don't know much about Databricks's Delta Lake, but it seems like it should be possible with Delta Tables?
Fix the extraction job
Is our oroblem is solved if we can make our extracts contain only what has changed since the previous extract?
Example
Suppose the following three files are in a folder of a data lake. (Gist with the 3 csvs and desired table outcome as csv).
I added the Extracted column in case parsing the timestamp from the filename is too tricky.
2020-09-14_CRM_extract.csv
| OppId | CustId | Stage | Won | LastModified | Extracted |
|-------|--------|-------------|-----|--------------|-----------|
| 1 | A | 2 - Qualify | | 9/1 | 9/14 |
| 2 | B | 3 - Propose | | 9/12 | 9/14 |
2020-09-15_CRM_extract.csv
| OppId | CustId | Stage | Won | LastModified | Extracted |
|-------|--------|-------------|-----|--------------|-----------|
| 1 | A | 2 - Qualify | | 9/1 | 9/15 |
| 2 | B | 4 - Closed | Y | 9/14 | 9/15 |
| 3 | C | 1 - Lead | | 9/14 | 9/15 |
2020-09-16_CRM_extract.csv
| OppId | CustId | Stage | Won | LastModified | Extracted |
|-------|--------|-------------|-----|--------------|-----------|
| 1 | A | 2 - Qualify | | 9/1 | 9/16 |
| 2 | B | 4 - Closed | Y | 9/14 | 9/16 |
| 3 | C | 2 - Qualify | | 9/15 | 9/16 |
End Result
Below is SCD-II for the three files as of 9/16. SCD-II as of 9/15 would be the same but OppId=3 has only one from valid_from=9/15 and valid_to=null
| OppId | CustId | Stage | Won | LastModified | valid_from | valid_to |
|-------|--------|-------------|-----|--------------|------------|----------|
| 1 | A | 2 - Qualify | | 9/1 | 9/14 | null |
| 2 | B | 3 - Propose | | 9/12 | 9/14 | 9/15 |
| 2 | B | 4 - Closed | Y | 9/14 | 9/15 | null |
| 3 | C | 1 - Lead | | 9/14 | 9/15 | 9/16 |
| 3 | C | 2 - Qualify | | 9/15 | 9/16 | null |
Interesting concept and of course it would a longer conversation than is possible in this forum to fully understand your business, stakeholders, data, etc. I can see that it might work if you had a relatively small volume of data, your source systems rarely changed, your reporting requirements (and hence, datamarts) also rarely changed and you only needed to spin up these datamarts very infrequently.
My concerns would be:
If your source or target requirements change how are you going to handle this? You will need to spin up your datamart, do full regression testing on it, apply your changes and then test them. If you do this as/when the changes are known then it's a lot of effort for a Datamart that's not being used - especially if you need to do this multiple times between uses; if you do this when the datamart is needed then you're not meeting your objective of having the datamart available for "instant" use.
Your statement "we have a DW as code that can be deleted, updated, and recreated without the complexity that goes along with traditional DW change management" I'm not sure is true. How are you going to test updates to your code without spinning up the datamart(s) and going through a standard test cycle with data - and then how is this different from traditional DW change management?
What happens if there is corrupt/unexpected data in your source systems? In a "normal" DW where you are loading data daily this would normally be noticed and fixed on the day. In your solution the dodgy data might have occurred days/weeks ago and, assuming it loaded into your datamart rather than erroring on load, you would need processes in place to spot it and then potentially have to unravel days of SCD records to fix the problem
(Only relevant if you have a significant volume of data) Given the low cost of storage, I'm not sure I see the benefit of spinning up a datamart when needed as opposed to just holding the data so it's ready for use. Loading large volumes of data everytime you spin up a datamart is going to be time-consuming and expensive. Possible hybrid approach might be to only run incremental loads when the datamart is needed rather than running them every day - so you have the data from when the datamart was last used ready to go at all times and you just add the records created/updated since the last load
I don't know whether this is the best or not, but I've seen it done. When you build your initial SCD-II table, add a column that is a stored HASH() value of all of the values of the record (you can exclude the primary key). Then, you can create an External Table over your incoming full data set each day, which includes the same HASH() function. Now, you can execute a MERGE or INSERT/UPDATE against your SCD-II based on primary key and whether the HASH value has changed.
Your main advantage doing things this way is you avoid loading all of the data into Snowflake each day to do the comparison, but it will be slower to execute this way. You could also load to a temp table with the HASH() function included in your COPY INTO statement and then update your SCD-II and then drop the temp table, which could actually be faster.
I have a super-class/subclass hierarchical relationship as follows:
Super-class: IT Specialist
Sub-classes: Databases, Java, UNIX, PHP
Given that each instance of a super-class may not be a member of a subclass and a super-class instance may be a member of two or more sub-classes, how would I go about implementing this system?
I haven't been given any attributes to assign to the entities so I find this very vague and I'm at a loss where to start.
To get started, you would have one table that contains all of your super-classes (in your example case, there would only be IT Specialist, but it could also contain things like Networking Specialist, or Digital Specialist). I've included these to give a bit more flavour:
ID | Name |
-----------------------------
1 | IT Specialist |
2 | Networking Specialist |
3 | Digital Specialist |
You also would have another table that contains all of your sub-classes:
ID | Name |
--------------------
1 | Databases |
2 | Java |
3 | UNIX |
4 | PHP |
For example, let's say that a Networking Specialist needs to know about Databases, and a Digital Specialist needs to know about both Java and PHP. An IT Specialist would need to know all four fields listed above.
There are two possible ways to go about this. One such way would be to set 'flags' in the sub-class table:
ID | Name | Is_IT | Is_Networking | Is_Digital
----------------------------------------------------
1 | Databases | 1 | 1 | 0
2 | Java | 1 | 0 | 1
3 | UNIX | 1 | 0 | 0
4 | PHP | 1 | 0 | 1
Keep in mind, this is only using a small number of skills. If you started to have a lot of super-classes, the columns in the sub-class table could get out of hand pretty quickly.
Fortunately, you can also use something known as a bridging table (also known as an associative entity). Essentially, a bridging table allows you to have two foreign keys that are primary keys in another table, solving the problem of a many-to-many relationship.
You would set this up by having a new table that associates which sub-classes belong with which super-classes:
ID | Sub-class ID | Super-class ID |
-------------------------------------
1 | 1 | 1 |
2 | 1 | 2 |
3 | 2 | 1 |
4 | 2 | 3 |
5 | 3 | 1 |
6 | 4 | 1 |
7 | 4 | 3 |
Note that there are 'duplicates' in both the sub-class ID and super-class ID fields, yet no duplicates in the ID field. This is because the bridging table has unique IDs, which it uses to make independent associations. Sub-class 1 (Databases) needs to be associated to two different groups (IT Specialist and Networking Specialist). Thus, two different associations need to be formed.
Both approaches above give the same 'result'. The only real difference here is that a bridging table will give you more rows, while setting multiple flags will give you more columns. Obviously, the way in which you craft your query will be different as well.
Which of the two approaches you choose to go with really depends on how much data you're dealing with, and how much scope the database is going to have for expansion in the future :)
Hope this helps! :)
I want to manage roles and permissions. Most of designs on web look like this
tables:
Users
Roles
UserRoles
Permissions
RolePermissions
Here, what is permissions? I am thinking for such a design instead:
Users
Roles
UserRoles
Permissions
In this design, Roles is supposed to be:
id | name
while permissions is supposed to be:
id | role_id | section | action
permissions defines which role in which section has what action control. something like this:
id | role_id | section | action
1 | 2 | posts | edit
2 | 2 | posts | add
3 | 2 | posts | delete
4 | 3 | users | approve
5 | 3 | users | edit
6 | 4 | articles | delete
7 | 2 | users | givepermission
It uses two strings instead of an extra table and numbers. Also checking it on PHP seems easier.
Does this design have problem? And is it recommended by you according to your experiences?
As to whether your design is recommended, then it depends on the problem you are trying to solve. However, your design makes sense for a RESTful application, and should be resilient.
For your permissions table, you may want to consider using a bitmask instead of strings.
There is a database of spare parts for cars, and online search by the name of spare parts. The user can type in the search, for example "safety cushion" or "airbag" - and the search result should be the same.
Therefore, I need somehow to implement the aliases for names of spare parts, and the question is how to store them in the database? Until now I have only one option that comes in mind - to create an additional table
| id | name of part | alias_id |
-------------------------------------------------- ---------------
| 1 | airbag | 10 |
| 2 | safety cushion | 10 |
And add additional field "alias_id" to table containing all the spare parts, and search by this field...
Are there other better options?
If I have understood correctly, it's best to have 3 tables in a many to many situation (if multiple parts have multiple aliases:
Table - Parts
| id | name of part |
-----------------------
| 1 | airbag |
| 2 | safety cushion |
Table - Aliases
| id | name of alias |
-----------------------
| 10 | AliasName |
Table - PartToAliases
| id | PartId | AliasId |
-------------------------
| 1 | 1 | 10 |
| 2 | 2 | 10 |
Your solution looks fine for the exact problem you described.
BUT what if someone writes safetycushion? or safety cuschion? With these kinds of variations your alias lookup table will soon become huge and and manualy maintaining these will not be feasible.
At that point you'll need a completely different approach (think full text search engine).
So if you are still sure you only need a couple of aliases your approach seems to be fine.
I start new e-commerce web application (pet project) that sale both t-shirt and shoe. My store has only free size T shirt so t-shirt has only color column while shoe has columns for size and color.
Now it's time to create table to store that data, I want to know is it good to create separate table for shoe and t-shirt or it's better to keep all of them in one table?
If it has a better idea to store such data, please let me know.
You definitely don't want to create a Shoe table and a TShirt table. Your shop might grow, and one day you'll have a thousand such product tables. Writing SQL for that would be a nightmare. Plus, you might have different kinds of t-shirts eventually, some with color, some with size and color, and so on. If you create a new table for each, you'll lose track of them quickly, and if you don't, why have separate tables for t-shirts and shoes, but not for one-size t-shirts and multi-size t-shirts?
While designing your database, you should be asking yourself: what are the entities in my realm? what are the things that never change and are uniquely identifiable? In a shop, a particular item that can be sold at a particular price is one such entity. So you might have a products table that has a key for each particular item you sell, and maybe a name, a type, a size and a color column:
item
id | type | name | size | color
------------------------------------
1 | shoe | Marathon | 9 | white
2 | shoe | Marathon | 9 | black
Looking at this table, you notice that we have two entries for the highly successful Marathon running shoe, and that seems to be a normalization violation. Indeed, you probably have two entities a shippable item and a catalog product. The shoe "Marathon" is probably something that has one picture and one description in your store, followed by a "available in the following colors and sizes:" line. So now you have two tables:
product
id | type | name | supplier | picture | description
--------------------------------------------------------------------------------------
1 | shoe | Marathon | TrackNField Co. | marathon.jpg | Run faster than light!
2 | tshirt | FlowerPower | SF Shirts | fpower.jpg | If you're going to San Francisco...
item
id | product_id | size | color | price
--------------------------------------
1 | 1 | 9 | white | 99.99
2 | 1 | 9 | black | 99.99
3 | 2 | | blue | 19.99
The "type" column in the product table can be a tricky one. You'll probably want to display products by category, let the user click on "shoes" and get all products with type "shoe". Easy so far, but eventually someone will mistype an entry "sheo", and then you can't find that product under shoes anymore. So it's better to separate the categorization from the products, for example by having a product_type table:
product_type
id | name
---------
1 | shoe
product
id | type_id | name | supplier | picture | description
--------------------------------------------------------------------------------------
1 | 1 | Marathon | TrackNField Co. | marathon.jpg | Run faster than light!
with a reference to the type in the product table. That's ok as long as your type hierarchy stays shallow, but what if you want to have subcategories, like "sneaker", "basketball shoe", "suede shoe", and so on? One shoe might even belong to several of these subcategories. In that case you can try this
category
id | name | supercategory_id
------------------------------------
1 | shoe |
2 | running shoe | 1
product_category
product_id | category_id
------------------------
1 | 2
product
id | name | supplier | picture | description
--------------------------------------------------------------------------------------
1 | Marathon | TrackNField Co. | marathon.jpg | Run faster than light!
And if you want to display multiple hierarchies of categorizations (as most big ecommerce sites do these days), you'll have to come up with something even more sophisticated.
Keep them all in one table and have a type field. The reason to do it this way is so that your data structure is scalable: i.e. if there is a new type of product then instead of adding a new table and having to drastically change your application code, you just use the same table and simply add a type.
if you don't want make it to complex you can keep all of them in the same table and create another table called "ProductType" that tells you if it is a shoe or a t-shirt.
The relationship will be One-To-Many on the "ProductType" side as you can have the same type of product associated with more then one record on the product table(where you store all your products)