In the TFP tutorial, the model output is Normal distribution. I noted that the output can be replaced by an IndependentNormal layer. In my model, the y_true is binary class. Therefore, I used an IndependentBernoulli layer instead of IndependentNormal layer.
After building the model, I found that it has two output parameters. It doesn't make sense to me since Bernoulli distribution has one parameter only. Do you know what went wrong?
# Define the prior weight distribution as Normal of mean=0 and stddev=1.
# Note that, in this example, the we prior distribution is not trainable,
# as we fix its parameters.
def prior(kernel_size, bias_size, dtype=None):
n = kernel_size + bias_size
prior_model = Sequential([
tfpl.DistributionLambda(
lambda t: tfd.MultivariateNormalDiag(loc=tf.zeros(n), scale_diag=tf.ones(n))
)
])
return prior_model
# Define variational posterior weight distribution as multivariate Gaussian.
# Note that the learnable parameters for this distribution are the means,
# variances, and covariances.
def posterior(kernel_size, bias_size, dtype=None):
n = kernel_size + bias_size
posterior_model = Sequential([
tfpl.VariableLayer(tfpl.MultivariateNormalTriL.params_size(n), dtype=dtype),
tfpl.MultivariateNormalTriL(n)
])
return posterior_model
# Create a probabilistic DL model
model = Sequential([
tfpl.DenseVariational(units=16,
input_shape=(6,),
make_prior_fn=prior,
make_posterior_fn=posterior,
kl_weight=1/X_train.shape[0],
activation='relu'),
tfpl.DenseVariational(units=16,
make_prior_fn=prior,
make_posterior_fn=posterior,
kl_weight=1/X_train.shape[0],
activation='sigmoid'),
tfpl.DenseVariational(units=tfpl.IndependentBernoulli.params_size(1),
make_prior_fn=prior,
make_posterior_fn=posterior,
kl_weight=1/X_train.shape[0]),
tfpl.IndependentBernoulli(1, convert_to_tensor_fn=tfd.Bernoulli.logits)
])
model.summary()
screenshot of the results executed the codes on Google Colab
I agree the summary display is confusing but I think this is an artifact of the way tfp layers are implemented to interact with keras. During normal operation, there will only be one return value from a DistributionLambda layer. But in some contexts (that I don't fully grok) DistributionLambda.call may return both a distribution and a side-result. I think the summary plumbing triggers this for some reason, so it looks like there are 2 outputs, but there will practically only be one. Try calling your model object on X_train, and you'll see you get a single distribution out (its type is actually something called TensorCoercible, which is a wrapper around a distribution that lets you pass it into tf ops that call tf.convert_to_tensor -- the resulting value for that op will be the result of calling your convert_to_tensor_fn on the enclosed distribution).
In summary, your distribution layer is fine but the summary is confusing. It could probably be fixed; I'm not keras-knowledgeable enough to opine on how hard it would be.
Side note: you can omit the event_shape=1 parameter -- the default value is (), or "scalar", which will behave the same.
HTH!
I'm working on a script that takes as input a sequence of MIDI notes and outputs a chord symbol for use in Impro-Visor, an open-source jazz improvisation helper. In order to take advantage of Impro-Visor's large vocabulary of chords I've been trying to add to music21's chord vocabulary--music21 itself will handle the MIDI pitches and the interpretation of most common chords--using the harmony.addNewChordSymbol method, but the system doesn't offer the new chords in its chord detection. For example, if I try this chord from the Harmony module's docs:
>>>harmony.addNewChordSymbol('BethChord', '1,3,-6,#9', ['MH', 'beth'])
>>>c = chord.Chord(['C3','D#3','E3','A-3'])
>>>print(harmony.chordSymbolFromChord(c))
'A-+/CaddD#'
Whereas I would hope in this case to get: 'Cbeth'
Music21 consistently suggests slash chords like the above rather than whatever chord I've tried to add to the vocabulary, presumably because the chord type to the left of the slash--'+', in this case--comes earlier in the OrderedDict in harmony.py. Is there any way to make the chord-detection prefer a custom chord type over these slash chords (which I don't have any way of handling)?
I found that just telling music21 you meant for "C" to be the root does the trick. (Otherwise, it will try to stack in thirds and treat "Ab" as the root.) Call .root() like this:
>>> harmony.addNewChordSymbol('BethChord', '1,3,-6,#9', ['MH', 'beth'])
>>> c = chord.Chord(['C3','D#3','E3','A-3'])
>>> c.root(c.bass())
>>> harmony.chordSymbolFromChord(c)
<music21.harmony.ChordSymbol CMH>
I work on theory tasks in quantum computing, and make simple experiments with Qiskit. Unfortunately, I can't find a way how to make a complex control gates there, where control is in the quantum register.
I would like to have a "c_if" analogue, which can be chained and use quantum bits as a control.
Smth like
swap(q1, q2).c_if(q0,Zero).c_if(q3,One)
Is there such an operation in the qiskit? How could I emulate such an operation if it doesn't exist?
Check out Qiskit documentation for the MCXGate, know as the Multi-controlled-X Gate. This gate lets you define how many control qubits you would like to include (perhaps the majority of your quantum register) and define a control state.
from qiskit import *
my_circuit = QuantumRegister(3,3)
my_circuit.append(circuit.library.MCXGate(2, ctrl_state='10'), [0,1,2])
Check out the documentation here.
There are also variations that will do Y gate Z gate or whatever you like depending if the circuit sees the correct control gate.
Thanks #Dulah for his answer. I found my old samples and they're working pretty fine with the 0.18.2 qiskit version.
from qiskit import QuantumRegister, ClassicalRegister, QuantumCircuit, execute, Aer
from qiskit.circuit.library.standard_gates.x import XGate, MCXGate
from qiskit.circuit.library.standard_gates.swap import SwapGate
simulator = Aer.get_backend('qasm_simulator')
qreg = QuantumRegister(4)
creg = ClassicalRegister(4)
qc = QuantumCircuit(qreg, creg)
control1 = XGate().control(3, None, '110') #: old-style multy-controlled qubits
#control1 = MCXGate(3, None, '110') # fashion-style multi-controlled qubits
control2 = SwapGate().control(2, None, '10')
qc.append(control1, [0, 1, 2, 3])
qc.append(control2, [0, 1, 2, 3])
qc.measure(qreg,creg)
job = execute(qc, simulator, shots=1000)
result = job.result()
counts = result.get_counts(qc)
print("\nTotal count for 00 and 11 are:",counts)
qc.draw()
The code gives me a result
Background: I'm working on a project that historically has relied on sparse matrices for a lot of the math, and developing a plugin to outsource some of the heavy lifting to theano. Since theano's sparse support is limited, we're building a dense version first -- but hopefully that explains why we're interested in the approach below.
The task: apply some operator to only the nonzero values of a matrix.
The following subroutine works most of the time:
import theano.sparse.basic as TSB
def _applyOpToNonzerosOfDense(self,op,expr):
sparseExpr = TSB.clean(TSB.csr_from_dense(expr))
newData = op(TSB.csm_data(sparseExpr)).flatten()
newSparse = TS.CSR(newData, \
TSB.csm_indices(sparseExpr), \
TSB.csm_indptr(sparseExpr), \
TSB.csm_shape(sparseExpr))
ret = TSB.dense_from_sparse(newSparse)
return ret
The problem comes when expr is not a canonical matrix tensor, but a row tensor (so, expr is 1xN and expr.broadcastable is (True, False)). When that happens, we need to be able to retain or restore the broadcast status in the returned tensor.
Some things I've tried that don't work:
dense_from_sparse doesn't support broadcastable settings
Theano 0.9 doesn't support assignment to ret.broadcastable
ret.dimshuffle( ('x',1) ) fails with "You cannot drop a non-broadcastable dimension."
ret has (ought to have) exactly the same shape as expr, so I wasn't expecting this to be hard. How do I get my broadcast settings back?
LOL, it's in the API: T.addbroadcast(x,*axes)
I have the following 11x11 linear system of equations Ax = b with:
A = {
{1.0000000000000000, 8.0000000000000000, 6.0000000000000000, 12.0000000000000000, 24.0000000000000000, 24.0000000000000000, 8.0000000000000000, 6.0000000000000000, 24.0000000000000000, 24.0000000000000000, 24.0000000000000000},
{4.5999999999999996, 41.8531411531233601, 33.0479488942856037, 87.8349057232554173, 149.3783917109033439, 195.3689938163366833, 121.0451669808013690, 48.8422484540841708, 223.6406089026404516, 851.8470736603384239, 269.3015780207464900},
{21.1599999999999966, 218.9606780479085160, 182.0278210198854936, 642.9142219510971472, 929.7459962556697519, 1590.3768227003254196, 1831.4915561762611560, 397.5942056750813549, 2083.9634145976574473, 30235.1432043200838962, 3021.8058301860087340},
{97.3359999999999701, 1145.5240206653393216, 1002.6076877338904296, 4705.8591727678940515, 5786.8317341801457587, 12946.2633183243797248, 27711.6501551604087581, 3236.5658295810949312, 19419.1186238102454809, 1073154.9275125553831458, 33907.3782725576675148},
{447.7455999999998539, 5992.9723163999815370, 5522.3546042079124163, 34444.8913989153879811, 36017.8173980603314703, 105387.4349242659372976, 419295.1650431178859435, 26346.8587310664843244, 180954.3130575636751018, 38090161.8577392920851707, 380471.2698060897528194},
{0.0000000000000000, 34.2801357124991952, 168.4702728821191613, 2101.6181209908259007, 1236.1435394200643714, 6631.0420254749351443, 38374.2674650820554234, 4069.0485156323466072, 28291.8793721561523853, 7044717.1197200166061521, 60211.4334496619121637},
{2059.6297599999993508, 31353.0895356311411888, 30417.0821226643129194, 252121.9823892920394428, 224178.4848274685500655, 857893.2134182706940919, 6344206.6583608603104949, 214473.3033545676735230, 1686197.1981563565786928, 1351958038.0734937191009521, 4269229.7229307144880295},
{0.0000000000000000, 179.3414198404317403, 927.9328280691040618, 15382.9524602928686363, 7693.8805767663707229, 53979.1670196200575447, 580627.4516345988959074, 33123.5797620395824197, 263633.8804078772664070, 250042569.2999326586723328, 675626.4184535464737564},
{0.0000000000000000, 938.2502198978935439, 5111.0461132262771571, 112596.6815912620077142, 47887.4794405465727323, 439410.6478194649680518, 8785268.3545934017747641, 269638.3520710353623144, 2456635.0642409822903574, 8874917956.1941699981689453, 7581135.8600852200761437},
{0.0000000000000000, 938.2502198978935439, 0.0000000000000000, 56298.3407956310038571, 23943.7397202732863661, 319571.3802323381532915, 8785268.3545934017747641, 0.0000000000000000, 269630.6777825467870571, 3293783983.7421655654907227, 1735440.7390556528698653},
{0.0000000000000000, 70.9608494071368625, 1546.2151390406352220, 34063.2210755480555235, 13279.8613116998949408, 129911.1650312914862297, 2657756.2850107550621033, 183537.2854802548536099, 1654054.3836708476301283, 5487391301.6329326629638672, 5049794.3807012736797333}
};
b = {1, 6.167551546217714, 39.66265463865314, 267.9960092725794, 1918.2310370808632, 137.49061855461255, 14662.396462231256, 1216.4598834815756, 11424.520672986631, 3808.17355766221, 6082.299417407878};
The matrix is clearly ill-conditioned, although the correct solution can be found with mathematica:
x = {0.0775277, 0.00771443, 0.087553, 0.0208838, 8.47931*1e-7, 0.00197285, 0.0000611365, 0.00187375, 0.000283606, 3.82771*1e-9, 0.000788588};
I now want to solve the system using this and many other similar matrices inside a C program.
I have tried almost every lapack function for solving a linear system of equations, in particular:
dgesv
dsgesv
dgels
dgelss
dgelsy
but they all give severely wrong results.
At this point I don't expect to have any typo / mistake from a programming point of view, since trying with well-conditioned matrices I get correct results.
I guess it's something conceptually or maybe I have to use other tools.. Is there anything I can do to find get the correct solution with some routine from mathematical libraries?
Solving ill-coditioned linear equations is generally hard. At least you could not use those one-step LAPACK APIs to get an answer with satisfied numerical error.
As a good start, you could use truncated SVD method to get a more numerically stable result.
https://en.m.wikipedia.org/wiki/Linear_least_squares_(mathematics)
This method is the most computationally intensive, but is particularly useful if the normal equations matrix, XTX, is very ill-conditioned (i.e. if its condition number multiplied by the machine's relative round-off error is appreciably large). In that case, including the smallest singular values in the inversion merely adds numerical noise to the solution. This can be cured with the truncated SVD approach, giving a more stable and exact answer, by explicitly setting to zero all singular values below a certain threshold and so ignoring them, a process closely related to factor analysis.
More effective methods may involve making the matrix well-conditioned before solving by finding a pre conditioning matrix. You need to have some understanding on the structure of the original matrix. You could find some more ideas in the following discussion.
https://www.researchgate.net/post/How_can_I_solve_an_ill-conditioned_linear_system_of_equations