Strange pattern of non-convergence when flattening in mixed effects logistic regression - logistic-regression

I'm running a simulation study on the effects of adding fractional numbers of successes and failures, which I'll call C, to mixed effects logistic regressions. I've simulated 2000 datasets and modeled each with 5 logistic regressions (adding an C of either 1, .5, .25, .1 and .05). The models converge on the majority of the datasets, but ~200 fail to converge when I add an C of .25 and ~50 fail to converge when I add an C of .5 (Sometimes I get a warning message and sometimes I get implausible standard errors). I very rarely see any evidence of non-convergence with the other values (I've looked at warning messages, standard errors and the ratio of highest to lowest eigenvalues in the random effects matrix). Even in the datasets that fail to converge when C = .25, slightly changing C often solves the problem, such as in this example (data sets available here: https://www.dropbox.com/sh/ro92mtjkpqwlnws/AADSVzcNvl0nnnzCEF5QGM6qa?oref=e&n=19939135)
m7 <- glmer(cbind(Data + .25, (10+.5- (Data + .25))) ~ Group*Condition + (1 + Condition |ID), family="binomial", data=df2)
Warning messages:
1: In eval(expr, envir, enclos) : non-integer counts in a binomial glm!
2: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model is nearly unidentifiable: very large eigenvalue
- Rescale variables?
summary(m7)
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
Family: binomial ( logit )
Formula: cbind(Data + 0.25, (10 + 0.5 - (Data + 0.25))) ~ Group * Condition + (1 + Condition | ID)
Data: df2
AIC BIC logLik deviance df.resid
7001.1 7040.0 -3493.6 6987.1 1913
Scaled residuals:
Min 1Q Median 3Q Max
-3.5444 -0.6387 0.0143 0.6945 2.9802
Random effects:
Groups Name Variance Std.Dev. Corr
ID (Intercept) 0.26598 0.5157
Condition 0.06413 0.2532 0.66
Number of obs: 1920, groups: ID, 120
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 1.760461 0.001226 1436.5 <2e-16 ***
Group -1.816952 0.001225 -1483.0 <2e-16 ***
Condition -0.383383 0.001226 -312.7 <2e-16 ***
Group:Condition -0.567517 0.001225 -463.2 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) Group Condtn
Group 0.000
Condition 0.000 0.000
Group:Cndtn 0.000 0.000 0.000
m8 <- glmer(cbind(Data + .2, (10+.4- (Data + .2))) ~ Group*Condition + (1 + Condition |ID), family="binomial", data=df2)
Warning message:
In eval(expr, envir, enclos) : non-integer counts in a binomial glm!
summary(m8)
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
Family: binomial ( logit )
Formula: cbind(Data + 0.2, (10 + 0.4 - (Data + 0.2))) ~ Group * Condition + (1 + Condition | ID)
Data: df2
AIC BIC logLik deviance df.resid
6929.3 6968.2 -3457.6 6915.3 1913
Scaled residuals:
Min 1Q Median 3Q Max
-3.5724 -0.6329 0.0158 0.6945 2.9976
Random effects:
Groups Name Variance Std.Dev. Corr
ID (Intercept) 0.2698 0.5194
Condition 0.0652 0.2553 0.66
Number of obs: 1920, groups: ID, 120
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 1.76065 0.07850 22.429 < 2e-16 ***
Group -1.81762 0.10734 -16.933 < 2e-16 ***
Condition -0.38111 0.06377 -5.977 2.28e-09 ***
Group:Condition -0.57033 0.08523 -6.692 2.21e-11 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) Group Condtn
Group -0.732
Condition -0.033 0.025
Group:Cndtn 0.029 0.045 -0.758
As this is a simulation study, I'm not especially interested in making those models converge, but I'd like to understand why they're not converging. Does anybody have any ideas?

Related

Plotting logistic regression line

this is my first post ever here so I'm not quit sure what is the proper form to ask the question. I'm trying to put picture of the results but since its my first post, the website telling me that I need 10 positive post for some credibility so I think that my charts doesn't appear. Also, I'm french, not perfectly bilingual. Please, be indulgent, I'm open for all comments and suggestions. I really need this for my master's projet. Thank you very much!
I have two sets of arrays which contains thousands of values In one (x_1_3) is all the value of temperature and y_0_100 contain only 0's and 100's which are associated to every temperature in x_1_3 sorted.
x_1_3 = array([[ 2.02],
[ 2.01],
[ 3.08],
...,
[ 0.16],
[ 0.17],
[-2.12]])
y_0_100 = array([ 0., 0., 0., ..., 100., 100., 100.])
The 0 in y_0_100 represent solid precipitation and 100 represent liquid precipitation I just want to plot a logistic regression line across my values
(I also tried to put the values in a dataframe, but it didnt work)
dfsnow_rain
AirTemp liquid%
0 2.02 0.0
1 2.01 0.0
2 3.08 0.0
3 3.05 0.0
4 4.89 0.0
... ... ...
7526 0.78 100.0
7527 0.40 100.0
7528 0.16 100.0
7529 0.17 100.0
7530 -2.12 100.0
7531 rows × 2 columns
X = x_1_3
y = y_0_100
# Fit the classifier
clf = linear_model.LogisticRegression(C=1e5)
clf.fit(X, y)
# and plot the result
plt.figure(1, figsize=(10, 5))
plt.clf()
plt.scatter(X.ravel(), y, color='black', zorder=20)
X_test = np.linspace(-15, 15, 300)
loss = expit(X_test * clf.coef_ + clf.intercept_).ravel()
plt.plot(X_test, loss, color='red', linewidth=3)
ols = linear_model.LinearRegression()
ols.fit(X, y)
plt.plot(X_test, ols.coef_ * X_test + ols.intercept_, linewidth=1)
#plt.axhline(1, color='.5')
plt.ylabel('y')
plt.xlabel('X')
plt.xticks(range(-10, 10))
plt.yticks([0, 100, 10])
plt.ylim(0, 100)
plt.xlim(-10, 10)
plt.legend(('Logistic Regression Model', 'Linear Regression Model'),
loc="lower right", fontsize='small')
plt.tight_layout()
plt.show()
Chart results
When I zoom in I realise that my logistic regression line is not flat, its the line that curves in a very small range (see picture below)
Chart when it's zoomed
I would like something more like this :
Logistic regression chart i would like
What am i doing wrong here? I just want to plot a regression line across my values from y0 to y100

How can we convert a Signal-to-noise in decible?

suppose ,
we get a value of Signal-to-noise(SNR) is 255.
what is value of SNR in dB ?
Note:
we know,
if a value is given 30 dB.
the SNR value will be = 10^3 = 1000
SNR Calculation – Simple
If your signal and noise measurements are already in dB form, simply subtract the noise figure from the main signal: S - N. Because when you subtract logarithms, it is the same as dividing normal numbers. The difference of the numbers is the SNR. For example: you measure a radio signal with a strength of -5 dB and a noise signal of -40 dB. -5 - (-40) = 35 dB.
SNR Calculation – Complicated
To calculate SNR, divide the value of the main signal by the value of the noise, and then take the common logarithm of the result: log(S ÷ N). There’s one more step: If your signal strength figures are units of power (watts), multiply by 20; if they are units of voltage, multiply by 10. For power, SNR = 20 log(S ÷ N); for voltage, SNR = 10 log(S ÷ N). The result of this calculation is the SNR in decibels. For example, your measured noise value (N) is 1 microvolt, and your signal (S) is 200 millivolts. The SNR is 10 log(.2 ÷ .000001) or 53 dB.
info from https://sciencing.com/how-to-calculate-signal-to-noise-ratio-13710251.html
This question might be best on https://physics.stackexchange.com/ though

Bayesian Network

I have following bayesian network :
I was asked to find:
Value of P(b)
The solution
P(b) = ΣA={a,¬a} P(A)P(b|A)
= 0.1 × 0.5 + 0.9 × 0.8 = 0.77
and value of P(d/a)
The solution:
P (d|a) = ΣB={b,¬b} P (d|B)p(B|a)
= 0.9 × 0.5 + 0.2 × 0.5 = 0. 55
How did they come up with above formula?
What rule they have used to find marginal probability from bayesian network graph?
I understand basic joint probability distribution formula which is just product of individual probabilities given its parents.
Some explanation and resources relating to this will be helpful.
Thank you
I guess I found my answer.
It uses Marginal Probabilities.
The formula is:
P(X/Y) = ΣZ={all possible values of z} P(X/Y,Z)P(Z|Y)
Now you can easily find above two probabilities.

statsmodel logit results very different from R and Stata

I am fitting a logistic regression using SAT scores to predict a binary outcome - the bivariate correlation is 0.17. Stata and R (aod package) both give a logit coefficient of 0.004, but statsmodel (python) gives -0.0013 (I have tried both MLE and IRLS). There is no missing data, and the number of observations is exactly the same across all three platforms – it is the same .csv file being used in each case.
R:
Call:
glm(formula = df$outcome ~ df$sat, family = "binomial", data = df)
Deviance Residuals:
Min 1Q Median 3Q Max
-0.7527 -0.5911 -0.4778 -0.3406 3.0509
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -7.758e+00 6.274e-02 -123.7 <2e-16 ***
df$sat 4.151e-03 4.351e-05 95.4 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 257024 on 334357 degrees of freedom
Residual deviance: 245878 on 334356 degrees of freedom
AIC: 245882
Number of Fisher Scoring iterations: 5
Stata:
. logit outcome sat
Iteration 0: log likelihood = -128512.03
Iteration 1: log likelihood = -123233.13
Iteration 2: log likelihood = -122939.88
Iteration 3: log likelihood = -122939.1
Iteration 4: log likelihood = -122939.1
Logistic regression Number of obs = 334,358
LR chi2(1) = 11145.86
Prob > chi2 = 0.0000
Log likelihood = -122939.1 Pseudo R2 = 0.0434
------------------------------------------------------------------------------
outcome | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
sat | .0041509 .0000435 95.40 0.000 .0040656 .0042362
_cons | -7.75775 .0627402 -123.65 0.000 -7.880719 -7.634782
Statsmodel:
Optimization terminated successfully.
Current function value: 0.399258
Iterations 5
Logit Regression Results
==============================================================================
Dep. Variable: outcome No. Observations: 334358
Model: Logit Df Residuals: 334357
Method: MLE Df Model: 0
Date: Wed, 15 Jul 2015 Pseudo R-squ.: -0.03878
Time: 13:09:47 Log-Likelihood: -1.3350e+05
converged: True LL-Null: -1.2851e+05
LLR p-value: 1.000
==============================================================================
coef std err z P>|z| [95.0% Conf. Int.]
------------------------------------------------------------------------------
sat -0.0013 3.69e-06 -363.460 0.000 -0.001 -0.001
==============================================================================
Generalized Linear Model Regression Results
==============================================================================
Dep. Variable: outcome No. Observations: 334358
Model: GLM Df Residuals: 334357
Model Family: Binomial Df Model: 0
Link Function: logit Scale: 1.0
Method: IRLS Log-Likelihood: -1.3350e+05
Date: Wed, 15 Jul 2015 Deviance: 2.6699e+05
Time: 13:09:48 Pearson chi2: 3.50e+05
No. Iterations: 7
==============================================================================
coef std err z P>|z| [95.0% Conf. Int.]
------------------------------------------------------------------------------
sat -0.0013 3.69e-06 -363.460 0.000 -0.001 -0.001
==============================================================================

Correct way to get weighted average of concrete array-values along continous interval

I've been looking for a while onto websearch, however, possibly or probably I am missing the right terminology.
I have arbitrary sized arrays of scalars ...
array = [n_0, n_1, n_2, ..., n_m]
I also have a function f->x->y, with 0<=x<=1, and y an interpolated value from array. Examples:
array = [1,2,9]
f(0) = 1
f(0.5) = 2
f(1) = 9
f(0.75) = 5.5
My problem is that I want to compute the average value for some interval r = [a..b], where a E [0..1] and b E [0..1], i.e. I want to generalize my interpolation function f->x->y to compute the average along r.
My mind boggles me slightly w.r.t. finding the right weighting. Imagine I want to compute f([0.2,0.8]):
array --> 1 | 2 | 9
[0..1] --> 0.00 0.25 0.50 0.75 1.00
[0.2,0.8] --> ^___________________^
The latter being the range of values I want to compute the average of.
Would it be mathematically correct to compute the average like this?: *
1 * (1-0.8) <- 0.2 'translated' to [0..0.25]
+ 2 * 1
avg = + 9 * 0.2 <- 0.8 'translated' to [0.75..1]
----------
1.4 <-- the sum of weights
This looks correct.
In your example, your interval's length is 0.6. In that interval, your number 2 is taking up (0.75-0.25)/0.6 = 0.5/0.6 = 10/12 of space. Your number 1 takes up (0.25-0.2)/0.6 = 0.05 = 1/12 of space, likewise your number 9.
This sums up to 10/12 + 1/12 + 1/12 = 1.
For better intuition, think about it like this: The problem is to determine how much space each array-element covers along an interval. The rest is just filling the machinery described in http://en.wikipedia.org/wiki/Weighted_average#Mathematical_definition .

Resources