I'm very confused about this topic. Initially I planned my DB without any "sorting" fields and would just push values in array in the order that I wanted. Everything worked fine until one day one of my collections array got randomly shuffled.
Google gives me a lot of controversial information on whether MongoDB persists order in arrays or not.
To make it even more confusing $push has $sort modifier. I don't understand why would it be there unless arrays are actually stay persistent (but my practice tells me they are not!). Why would anyone want to push values in an array in certain order when it can be shuffled at any random time?
Also what's the best way to keep the order in DB? I ended up adding order field. But the problem is that my documents have a lot of nested (and sub nested) docs that need to be presented in a certain order. As I know, the only way to order items in nested array is to use aggregation: unwind -> sort -> group. But that looks like a lot of steps if I have quite a bit of fields that I need to sort. Perhaps would be better way to sort it after querying the document?
Alright, after a lot of googling and chatting with MongoDB Atlas support, I can confirm that arrays in MongoDB DO persist their order. Most likely we had a bug on application layer that messed up with arrays.
what is the closest thing to Arrays in Elixir. By arrays I mean, a container for values which I can access in constant time.
I've looked at tuple, but according to the documentation:
Tuples are not meant to be used as a “collection” type (which is also suggested by the absence of an implementation of the Enumerable protocol for tuples): they’re mostly meant to be used as a fixed-size container for multiple elements.
What I actually want to do:
I want to store n processes in an array and periodically pick a random process and send it a message.
I'm open to other suggestions too.
I ended up using a combination of list and registry since I was working with processes. I got many responses on Elixir forum which I'm listing below for future reference:
Tuple: stored continuous in memory, constant access time, editing results in copying whole structure. Does not implement Enumerable protocol.
linked-List: O(n) access time, prefixing is cheaper than suffixing. Implements Enumerable protocol.
Map: O(log n) read, write, delete time. Also implements Enumerable protocol.
:array: array module from Erlang.
registry: (applicable only if storing processes) A local, decentralized and scalable key-value process storage.
Also, note 2 and 3 (List and Map) are persistent data structures.
There are also two Elixir packages Arrays
and Tensor that provide similar functionalities.
Elixir has an array module via erlang: http://erlang.org/doc/man/array.html
Like with mapping in the Solidity language, Elixir has map().
I'm a student and starting to relearn again the basics of programming.
The problem I stated above starts when I have read some Facebook posts that most of the programmers use arrays in their application and arrays are useful. And I started to realize that I never use arrays in my program.
I read some books but they only show the syntax of array and didn't discuss on when to apply them in creating real world applications. I tried to research this on the Internet but I cannot find any. Do you guys have circumstance when you use arrays. Can you please share it to me so I can have an idea.
Also, to clear my doubts can you please explain to me why arrays are good to store information because database can also store information. When is the right time for me to use database and arrays?
I hope to get a clear answer because I have one remaining semester before the internship and I want to clear my head on this. I do not include any specific programming language because I know most of the programming language have arrays.
I hope to get an answer that can I can easily understand.
When is the right time for me to use database and arrays?
I can see how databases and arrays may seem like competing solutions to the same problem, but you're comparing apples and oranges. Arrays are a way to represent structured data in memory. Databases are a tool to store data on disk until you need to retrieve it.
The question you pose is kind of like asking: "When is the right time to use an integer to hold a value, vs a piece of paper?" One of them is a structural representation in memory; the other is a storage tool.
Do you guys have circumstance when you use arrays
In most applications, databases and arrays work together. Applications often retrieve data from a database, and hold it in an array for easy processing. Here is a simple example:
Google allows you to receive an alert when something of interest is mentioned on the news. Let's call it the event. Many people can be interested in the event, so Google needs to keep a list of people to alert. How? Probably in a database.
When the event occurs, what does Google do? Well it needs to:
Retrieve the list of interested users from the DB and place it in an array
Loop through the array and send a notification to each user.
In this example, arrays work really well because users form a collection of similarly shaped data structures that needs to be put through a similar process. That's exactly what arrays are for!
Some other common uses of arrays
A bank wants to send invoice and payment due reminders at the end of the day. So it retrieves the users with past due payments from the DB, and loops through the users' array sending notifications.
An IT admin panel wants to check whether all critical websites in a list are still online. So it loops through the array of domains, pings each one and records the results in a log
An educational program wants to perform statistical functions on student test results. So it puts the results in an array to easily perform operations such as average, sum, standardDev...
Arrays are also awesome at keeping things in a predictable order. You can be certain that as you loop forward through an array, you encounter values in the order you put them in. If you're trying to simulate a checkout line at the store, the customers in a queue are a perfect candidate to represent in an array because:
They are similarly shaped data: each customer has a name, cart contents, wait time, and position in line
They will be put through a similar process: each customer needs methods for enter queue, request checkout, approve payment, reject payment, exit queue
Their order should be consistent: When your program executes next(), you should expect that the next customer in line will be the one at the register, not some customer from the back of the line.
Trying to store the checkout queue in a database doesn't make sense because we want to actively work with the queue while we run our simulation, so we need data in memory. The database can hold a historical record of all customers and their checkout outcomes, perhaps for another program to retrieve and use in another way (maybe build customized statistical reports)
There are two different points. Let's me try to explain the simple way:
Array: container objects to keep a fixed number of values. The array is stored in your memory. So it depends on your requirements but when you need a fixed and fast one, just use array.
Database: when you have a relational data or you would like to store it in somewhere and not really worry about the size of the objects. You can store 10, 100, 1000 records to you DB. It's also flexible and you can select/query/update the data flexible. Simple way to use is: have a relational data, large amount and would like to flexible it, use database.
Hope this help.
There are a number of ways to store data when you have multiple instances of the same type of data. (For example, say you want to keep information on all the people in your city. There would be some sort of object to hold the information on each person, and you might want to have a data structure that holds the information on every person.)
Java has two main ways to store multiple instances of data in memory: arrays and Collections.
Databases are something different. The difference between a database and an array or collection, as I see it, are:
databases are persistent, i.e. the data will stay around after your program has finished running;
databases can be shared between programs, often programs running in all different parts of the world;
databases can be extremely large, much, much larger than could fit in your computer's memory.
Arrays and collections, however, are intended only for use by one program as it runs. Your program may want to keep track of some information in order to do its calculations. But the data will be in your computer's memory, and therefore other programs on other computers won't be able to access it. And when your program is done running, the data is gone. However, since the data is in memory, it's much faster to use it than data in a database, which is stored on some sort of external device. (This is really an overgeneralization, and doesn't consider things like virtual memory and caching. But it's good enough for someone learning the basics.)
The Java run time gives you three basic kinds of collections: sets, lists, and maps. A set is an unordered collection of unique elements; you use that when the data doesn't belong in any particular order, and the main operations you want are to see if something is in the set, or return all the data in the set without caring about the order. A list is ordered, though; the data has a particular order, and provides operations like "get the Nth element" for some number N, and adding to the ends of the list or inserting in a particular place in the list. A map is unordered like a set, but it also attaches keys to the data, so that you can look for data by giving the key. (Again, this is an overgeneralization. Some sets do have order, like SortedSet. And I haven't even gotten into queues, trees, multisets, etc., some of which are in third-party libraries.)
Java provides a List type for ordered lists, but there are several ways to implement it. One is ArrayList. Like all lists, it provides the capability to get the Nth item in the list. But an ArrayList provides this capability faster; under the hood, it's able to go directly to the Nth item. Some other list implementations don't do that--they have to go through the first, second, etc., items, until they get to the Nth.
An array is similar to an ArrayList, but it has a different syntax. For an array x, you can get the Nth element by referring to x[n], while for an ArrayList you'd say x.get(n). As far as functionality goes, the biggest difference is that for an array, you have to know how big it is before you create it, while an ArrayList can grow. So you'd want to use an ArrayList if you don't know beforehand how big your list will be. If you do know, then an array is a little more efficient, I think. Although you can probably get by mostly with just ArrayList, arrays are still fundamental structures in every computer language. The implementation of ArrayList depends on arrays under the hood, for instance.
Think of an array as a book, and database as library. You can't share the book with others at the same time, but you can share a library. You can't put the entire library in one book, but you can checkout 1 book at a time.
Good day!
I need to find a base for storage and processing complex structured information.
Something like a mind map. Need to have some arbitrary values in groups with connections to each other, connection must also have titles.
The biggest problem is that I need to get all the related values without knowing exactly what are the connections and how many of them.
For example:
With VALUE 3 connected
VALUE 1 from the group A as NAME OF COMMUNICATION 1
and VALUE 2 from group B as NAME OF COMMUNICATION 2
and ...
Before any level of the connections (i.e., the values of all properties connected to the associated properties, and for these properties and so on until a predetermined level) - but it can be implemented in the application logic.
I looked at some noSQL base, but they do not allow such requests without knowing the exact value or links. I pondered on the mysql development with a lot of logic in the application to handle all this, but perhaps there is a more suited storage for such a task?
I would be grateful for any help.
http://magika.tk/struct.png - A schematic example.
As Philipp says mind-maps are a type of graph, usually a spider diagram. A graph based NoSQL databases, such as Neo4j would be suitable. Here's a longer list. Graph databases store information about the nodes and the edges. Each node has a pointer to all its adjacent nodes so counting connections and groups should be very fast.
I need to store a graph for the map of a game inside a game server written in C.
The graph has ~200 nodes and 3 kinds of edges that can connect two nodes (these three kind can also overlap: a node can be connected by 2 edges of two different types for example). The maximum degree of a node is something like 5-6 nodes.
What I would like to have is to have this static structure implemented in an efficient way to allow me to do simple operations like
is n1 connected to n2? (with all kinds of edges in case of affermative response)
what is n1 connected to? (with all kinds of edges or a specific one)
but in a multi-threaded environment since there will be many instances of the game that relies on the same static graph.
Since the graph can't be modified and the structure is well known I'm sure there are some tricks to implement it in a cool fashion to use least resources possible.
I would like to avoid using STL or Boost for now.. do you have any clues on a data structure that could suit well?
(it's not a premature optimization, the fact is that it will run on a vps and I don't have many ram neither cpu power so I need to keep it tight)
EDIT: just because I forgot (and thanks to make me realize it) the graph is undirected so every edge is symmetric..
Thanks in advance
Many answers are possible. This one relies on the fact that you have relatively few nodes. The advantage of this approach is probably unbeatable performance.
The idea is to represent your graph as a 200x200 matrix of bytes, each entry representing an edge. The byte gives you 256 different possible values, where a 0 will obviously mean "no connection" and any non-zero combination of bits can represent up to 8 different edge types.
Let the "row" of this matrix be the starting node and the "column" be the destination. Initialize the structure such that for every edge connecting one node with another, there's a value at the intersection of starting / ending. That value can be a combination of bits representing edge types.
To find out whether one node connects to another, simply query the byte at the intersection of one node and the other: If there's a nonzero value there, then there is a connection, and the value will tell you what kind.
For 200 nodes, this data structure will eat up 40 KB, which is pretty moderate. It won't scale too well once you get beyond, say, 1000 nodes.
As long as nothing (apart from one-time initialization) ever writes to this structure, it will be naturally thread safe, as its state never changes.
Since degrees are limited, you can get very good performance by just representing a node by a struct with arrays of pointers to other nodes (one array for each edge type).
Regardless of the data structure you pick, you can avoid worrying about multithreading if your graph is read-only (OK for multiple thread to access it without synchronization).