Multiple Actions in DQN (binary action vector) - artificial-intelligence

I have a requirement to have multiple actions as the ouput for one of the RL use case.
Multi-Actions as output - possible values are 0 or 1 .
Binary Action Vector ( 1 1 1 1 0 1 1 0 1 1)
I am wondering how to implement multiple actions as the output using DQN ? Is it possible with DQN ? If yes, Please provide as much detail and explanation as possible as I am just a beginner.
Thank You

Related

TSQL List all the constituent bitwise integers within a larger integer

Firstly, apologies for not using the correct terminology here. I don't actually know the correct terms, and thus have failed miserably to find a solution. Please accept my example as the question, and I'll update the question accordingly if somebody can enlighten me (or delete and read the actual solution where it exists).
So far my research has only yielded comparisons of single bits within the whole, rather than showing all.
Given a set of integers:
ELEMENT
-----------
Bricks 1
Plaster 2
Cement 4
Concrete 8
I have a result set that provides how these materials are used:
MIXTURE ELEMENTS
----------------------
MixtureFoo 3
MixtureBar 7
MixtureBaz 11
I need to show the final set of mixtures, but with each constituent element listed that is used in the respective mixture:
MIXTURE ELEMENTS ELEMENT
------------------------------
MixtureFoo 3 1
MixtureFoo 3 2
MixtureBar 7 1
MixtureBar 7 2
MixtureBar 7 4
MixtureBaz 11 1
MixtureBaz 11 2
MixtureBaz 11 8
You could use bitwise operations:
SELECT *
FROM t
JOIN ELEMENT e
ON t.ELEMENTS & e.w = e.w
ORDER BY MIXTURE, w;
db<>fiddle demo

Array operations in Matlab with different array sizes: different behavior depending on the Matlab version?

Sorry if this has been asked before, I'm new to the site and tried looking for a similar question but couldn't find anything on the site or in the MATLAB docs.
So I have the following MWE:
a = ones(5,3);
b = ones(5,1);
a == b
a ./ b
If I run this in my laptop using Matlab R2018a, the operation goes through with the following results:
ans =
5×3 logical array
1 1 1
1 1 1
1 1 1
1 1 1
1 1 1
ans =
1 1 1
1 1 1
1 1 1
1 1 1
1 1 1
So basically it will interpret the second array as a column vector and compare/divide the elements of each column in the first matrix with/by the corresponding elements of the column vector.
However, if you run this exact same code on version R2015b (version I have in a separate workstation) it throws the following error:
Error using ==
Matrix dimensions must agree.
And a similar one for ./
Does this mean the behavior of basic math operations for arrays was changed in one of the latest versions of Matlab? I haven't been able to locate such a change in the docs. Is there any way to get the behavior of the first example on older versions?
This behaviour changed in R2016b. See this blog post for details: https://blogs.mathworks.com/loren/2016/10/24/matlab-arithmetic-expands-in-r2016b/
There is no way to get the same implicit expansion behaviour in older versions, although you can use bsxfun to get explicit expansion.

Johnson-Trotter Permutation

I am trying to write a program that can generate permutations using the Johnson-Trotter method with varying number of elements. I am still confused on how to get the permutations exactly. For 5 elements, I can only get this far and then I get stuck. I am not asking for all of them just a few more so I can get the pattern down.
1 2 3 4 5
1 2 3 5 4
1 2 5 3 4
1 5 2 3 4
5 1 2 3 4
I found the following page:
http://introcs.cs.princeton.edu/java/23recursion/JohnsonTrotter.java.html
Try running this Java program step by step, it will surely be of great help...
I have used the Johnson-Trotter algorithm in a card-melding game. The princeton.edu link mentioned above is widely quoted, but pretty confusing. And the code I've seen is either recursive (yuck) or inefficient. I rewrote it as an iterator, and it works great in my game. See my other post here: https://stackoverflow.com/a/28241384/4266886 with (very short) code. Let me know if I can help further.

pattern recognition - "is this a pattern?"

I have a large vector of numbers, say 500 numbers. I would like a program to detect patterns (reoccurrence in this case) in such vector based on following rules:
A sequence of numbers is a pattern if:
The size of the sequence is between 3 and 20 numbers.
The RELATIVE positions of the numbers in sequence is repeated at
least one other time in a vector. So let's say if I have a sequence
(1,4,3) and then (3,6,5) somewhere else in the vector then (1,4,3) is
a pattern. (as well as (2,5,4), (3,6,5) etc.)
The sequences can't intersect. So, a vector (1,2,3,4,5) does not
contain patterns (1,2,3) and (3,4,5)(we can't use the same number for
both sequences). However, (1,2,3,3,4,5) does contain a pattern
(1,2,3) (or (3,4,5))
A subset A of a pattern B is a pattern ONLY IF A appears somewhere
else outside B. So, a vector (1,2,3,4,7,8,9,2,3,4,5) would contain
patterns (1,2,3,4) and (1,2,3), because (1,2,3,4) is repeated (in a
form of (2,3,4,5)) and (1,2,3) is repeated (in a form (7,8,9)).
However, if the vector was (1,2,3,4,2,3,4,5) the only pattern will
be (1,2,3,4), because (1,2,3) appeares only in context of (1,2,3,4).
I'd like to know several things:
First of all I hope the rules don't go against each other. I made them myself so there might be a clash somewhere that I didn't notice, please let me know if you do notice it.
Secondly, how would one implement such system in the most efficient way? Maybe someone can point out towards some particular literature on the subject? I could go number by number starting with searching a sequence repetition for all subsets of 3, then 4,5 and till 20. But that seems to be not very efficient..
I am interested in implementation of such system in C, but any general guidance is very welcome.
Thank you in advance!
Just a couple of observations:
If you're interested in relative values, then your first step should be to calculate the differences between adjacent elements of the vector, e.g.:
Original numbers:
1 4 3 2 5 1 1 3 6 5 6 2 5 4 4 4 1 4 3 2
********* ********* ********* *********
Difference values:
3 -1 -1 3 -4 0 2 3 -1 1 4 3 -1 -3 0 -3 3 -1 -1
****** ****** ****** ******
Once you've done that, you could use an autocorrelation method to look for repeated patterns in the data. This can be computed in O(n log n) time, and possibly even faster if you're only concerned with exact matches.

how to solve this with kmeans clustering and use cosine similiraty

can anyone tell to me how k-means clustering working on textmining..
and i using cosine similarity as the distance metric.
nim 310910022 320910044 310910043 310910021
access 0 2 3 5
abdi 1 0 0 0
actual 5 0 0 1
arrow 0 1 1 2
this data is on listview
How can I do that in VB.net? to get any cluster and trending topic of the term ?
Thank in advance
First I would separate the problem into two parts:
Computing the k-means clustering assignments
Getting the data from the GUI (you mention the data is on a listview)
I assume 2 is straight forward and you just need help on 1.
I would start by re-writing the code to just read a TSV text file of the data as you specified. This will make things a lot easier to debug.
Then ask if you want to implement the kmeans algorithm yourself or use a library.
If you want to implement it, here is a link to the pseudo code
http://www.scribd.com/doc/89373376/K-Means-Pseudocode
You can also search up other kmeans pseudocode.
If you want to use a library to just "run" your data against a kmeans algorithm, here is one example in python/scipy.
http://glowingpython.blogspot.com/2012/04/k-means-clustering-with-scipy.html
Regardless of which approach you use, realize that kmeans is non-deterministic and you may get different answers everytime you run it. I would recommend computing against a known validation set to see if the data is roughly what you think it should be.

Resources