I am trying to convert a large database (~3m rows) that contains the following data set titled "Posts":
+-------|---------------|-----------------------+
| id | name | tags |
|-------|---------------------------------------|
| 1 | post title | tag_a, tag_b |
| 2 | another title | tag_b, tag_e, tag_j |
+-------|---------------|-----------------------+
I also have an empty "tags" table with the headings id, title and a "posts_tags" table with the headings id, post_id, tag_id
Post <-- Habtm --> Tag
My question:
I would like to know the most efficient (preferred but not required cake way) of populating the "tags" table and the "posts_tags" habtm table while keeping the tags table free from duplicates?
Many Thanks SO Team!
I have no time to write code right now.
You could get all posts (I recommend you paginate your result) and for each post, you get its tags and explode it by comma.
Then you create an HABTM array data using the tags and the currently post, and finally you save your data.
Related
I am getting started with Dynamodb and I am trying to figure out what is the correct way to structure the following:
I have a user, and each user can have multiple pictures (s3 links and some metadata), with no limit to the amount.
Whenever I am calling for a user I would retrieve all their pictures, there would be no reads on a single picture, and each time a user uploads a picture, I need to store it for the user.
In Mongodb I would have created an array called pictures holding objects with each picture's data. Is this also the correct approach in Dynamodb?
Here is how you can do with dynamodb
| pk | sk |
| user1 | metadata | age:24 | name: Jon| ...
| user1 | picture#1234#id1 | url:abc.com/xyz.jpeg | ... some other metadata
| user1 | picture#2456#id2 | url:abc.com/xyz.jpeg | ... some other metadata
| user1 | picture#4567#id3 | url:abc.com/xyz.jpeg | ... some other metadata
where 1234, 2456 may represent epoch time uploading the picture.
Now you can do queries like
get me all the pictures in chronological order.
Select * from table where pk=user1 and sk starts with picture#
get me all the pictures uploaded b/w or after certain date.
Select * from table where pk=user1 and sk starts with picture# and sk>picture#1234
Get me user details
Select from table where pk=user1 and sk=metadata
This will make sure few things
No race condition/lock while uploading the picture
you can decide how many images you want to fetch, And you will read only what you need, instead of loading all of them and then filtering.
I am building an eCommerce site and I have about 10 different custom post types. For each custom post type, I have about 20 custom fields (with a single meta_value assigned to each meta_key).
What I was thinking about doing is grouping all 20 custom fields into a single custom field so that I'll have 20 meta_values (array) assigned to a single meta_key. This would essentially reduce the number of rows inside the database table from 20 rows down to a single row for each custom post type.
For example, let's say I have post type "book".
Currently my post_meta table for this post type might look like this:
meta_key | meta_value
_________________________
chapters | 5
_________________________
paragraphs | 10
_________________________
author | john_smith
_________________________
price | 19
_________________________
If I convert it to this:
meta_key | meta_value
____________________________
book_detail | [chapters:5 ,paragraphs:10, author:john_smith, price:19]
____________________________
Would doing something like this optimize WordPress query speed or would I just be wasting time?
The tables should be kept as normalized as possible. Storing the CSV or any delimited values is not recommended.
Also, in case you need to query a specific aspect of a post such as only chapters, then normalized data comes handy where you just need to put a simple predicate:
where metakey = 'chapters';
I have a dataset like this, and I'm looking for a way to add a category, based on what kind of product I have.
Can I search for Apple + Orange and assign them to a category named Fruits, and similar with Milk + Wine and assign them to another category, named Drinks?
| Item | Category |
|-------|----------|
| Apple | | <-- Fruits
| Orange| | <-- Fruits
| Milk | | <-- Drinks
| Wine | | <-- Drinks
Or maybe a simpler method: find any rows containing Milk and assign them to category Drinks?
This is something you can do without code.
Filter or facet in the Item field for each value
Create a facet on the Category field
Click the edit button next to the blank value in the Category facet and type with the category you want to add.
Edit your Item facet or filter to move to the next category and repeat this process until you have categorized all yours items.
As magdmartin says you can do this using facets and edits - the solution he describes is probably the simplest approach and least error prone. However, if you prefer to do in a single step you can use GREL to test the content of the Item cell and then set the value in the Category cell dependent on the content of the Item cell.
with(cells["Item"].value.toLowercase(),w,if(or(w=="orange",w=="apple"),"Fruits",if(or(w=="milk",w=="wine"),"Drinks","")))
This is the same approach as given by Ettore Rizza above but in GREL rather than Jython.
magdmartin and Owen Stephens give good answers. Another simple way using GREL:
From the options dropdown for your 'Item' choose Edit column > Add column based on this column...
New column name 'Category' and in the expression set:
value.replace("Apple","Fruit").replace("Orange","Fruit").replace("Milk","Drink").replace("Wine","Drink")
You could keep adding .replace("whatever food","whatever category") ad nauseum
I am creating a site that allows users to add a number of various items to their pages. These items can be very different from each other, such as text, images, numbers, etc. To handle this, I have one table that stores one entry for each item. The table has an id, the location of the item on the page, and the type of the item. I then have one table for each type of item which holds the more specific information about that item.
What I want to do is to set up an association so that when the page loads, it grabs the items from the main table in order - which is simple - and then also grabs the associated information from the proper table of specific information.
An example main_table could be:
id | location | type
1 | 1 | text
2 | 2 | image
3 | 3 | plot
4 | 4 | question
The table for text items may look like:
id | text |
1 | aaa |
2 | bbb |
The table for plots may look like:
id | type | datasets |
1 | bar | 3
2 | scatter | 2
So, can I set up an association where the main_data model pulls in the one associated file from the proper table? I could set up a separate model that selected the table for the specific information based on a passed in variable and set the source, but I am not sure how you would pass the variable with the type information to the model, nor am I sure if I am approaching this properly?
Thanks
PS
I had thought of the obvious - to combine the items into one table and simply have empty fields for everything not related to that item for each entry. So a text entry would still have columns for plot stuff, but they wouldn't be filled, etc. However, I wanted to be less wasteful of space - unless everyone thinks it would work better.
Sorry I don't have the reputation to comment for clarification, but this is what I made of your question. I think you're overthinking it...
The one thing that sticks out about your question, is that things are going to get very messy with the way you're approaching it.
In my opinion the best way to handle this is to give each item type it's own table then associations in the model definitions.
This is how your tables will look like, remember CakePHP tables need to be plural and user_id is deliberately titled as such.
Users
id | username
1 | example
Questions
id | user_id | text
1 | 1 | this is a question
Plots
id | user_id | text
1 | 1 | this is plot
This in your app/Model folder you will need files called User.php & Question.php & Plot.php (singular naming convention) that should look something like this
User.php
<?php
public $hasMany = array(
'Question',
'Plot'
);
?>
Question.php
<?php
public $belongsTo = 'User';
?>
Plot.php
<?php
public $belongsTo = 'User';
?>
Once you have set up the above, you will be able to easily access Plots and Questions in your views, I've provided below a snippet of a potential users controller and expected usage you can expect to use... learn to use debug() in CakePHP it's fantastic.
Controller/UsersController.php
<?php
...
public function index() {
$this->set('users', $this->paginate());
}
...
?>
View/Users/index.ctp
<div>
<?php
foreach ($users as $user):
debug($user);
echo $user['Question']['text'];
echo $user['Plot']['text'];
endforeach;
unset($user);
?>
</div>
After reading your question again, you mention "order" and "location". You can handle this by adding a column for "date_created" and "page_position" to each different table. I still feel like you overcomplicated your solution, and I hope I didn't miss something.
please help me on this one:)
Most of the time I am creating static pages like Contact Us, About Us, FAQs etc
If I want to store some dynamic items in the page, how will my table look like?
Let's say for the FAQ page, I am to store the question and the answer on the database. For the contact page, I will store emails and and some other contact information. Does that mean I have to create a separate table for each?
Often times I've seen people create a table for Meta information. It resembles a key => value relationship, where the first field of a row is the name and the second is the value. So if you were to manage your global contact information in the database, you may have the following rows:
Meta-Data Table
Meta ID | MetaTitle | Meta Value
---------------------------------------------------------
01 | email_address | some.guy#somedomain.com
02 | phone_number | 1.234.567.8901
03 | num_subscribers | 2312
04 | page_styles | background-color:#333333;color:#ffffff
As for your frequently asked questions, you could do that as a table, if you like:
Questions
Question ID | Question | Answer
-------------------------------------------
01 | How tall are you? | Not nearly tall enough.
Or you could simply create a generic 'pages' table:
Pages
Page ID | Page Title | Page Content
--------------------------------------------
01 | FAQs | How tall are you?
| | Not nearly tall enough.
--------------------------------------------
02 | Contact Us | Phone: 1.234.567.8901
| | Email: some.guy#somedomain.com
Based on the information provided, yes - it's quite likely you'll need separate tables.
Tables are just groupings of similar information, and email doesn't relate to questions & answers. With more information about what you're looking to do, and any business rules around it, we can help you with structure if you'd like.
You could just have 1 table which has a page id column and a page content column. The page id being different for each page you want to store in the database. And the page content column can be a text field which contains the HTML you want to display on that page. Then on each page you would query the database passing the parameter for the page content you wanted to display.