help sdmksubs

Title

sdmksubs -- Create substitution matrices for sequence analysis

Syntax

sdmksubs [if] [in], SUBSmat(string) [STATEvar(varname) IDvar(varname) DTYPE(string) RCB(string) RAW DROPDiag LAG(integer)]

options

Required
subsmat(string) names the matrix in which to store the substitution costs

dtype(string) Type of substitution cost to calculate

Optional
statevar(varname) State variable (mandatory unless type is "flat" or "linear")

idvar(varname) ID variable (mandatory unless type is "flat" or "linear")

NStates(real) Number of states (relevant to "flat" and "linear" types only)

rcb(string) "r", "c" or "b" for row, column or both as input to cost measure

raw If omitted, scale substitution costs to 0-1

dropdiag Drop the diagonal from the calculations

lag Lag (defaults to 1)

Description

sdmksubs creates substitution matrices for use with SADI
tools such as oma, sdhamming and twed. The dtype
option controls the type of matrix. For types other than "flat" and
"linear", the idvar option is necessary to identify blocks of
sequences. The rcb option controls whethers row or column
summaries are used (or both, combining them in a Euclidean fashion). The
raw option prevents re-scaling of the substitution matrix to a
maximum of 1.0. The dropdiag option drops the diagonal when
calculating the substitutions costs. By default, transitions are from
t-1 to t, but the lag option allows the specification of larger
values. If the state is observed at small time intervals, with a
consequently high tendency for cases to remain in the same category,
increasing the lag will give a transition matrix with more data
off-diagonal, and may be preferable to using the dropdiag option

Types

The type of substitution matrix is controlled by the dtype option.

Two matrix types do not depend on the data:

flat: all states are equally different (all ones except diagonal)

linear: states are on a linear scale, and differences are the absolute difference in value

The bulk of matrix types depend on the transition table implied by the data (and require the idvar option):

t2s: The conventional 2 - p_ - p_

pct: like t2s but use all the transition probabilities in rows/cols i and j

pearson: based on the Pearson residuals in rows/cols i and j

relres: based on the relative residuals in rows/cols i and j

chisq: based on the chi-square difference for rows/cols i and j

Data

The sequence data must be in long format with one
variable indicating the state, and an ID variable identifying sequences.
The transition table of t-1 to t is used to create the data-based
substitution matrices. If the dropdiag option is in force, the
diagonal of this table is set to zero (this tends to increase the
variability in the substitution matrix, particularly when transitions
tend to happen much less than once per time-unit). If the lag
option is in force, transitions between t-lag and t are used.

Rows, columns or both

Traditionally, the transition-based substitution matrix was based
on outflow (row) proportions, D_ = 2 - p_ - p_ where p_ =
n_/n_. There is no particular reason that inflow proportions
could not be used (p_ = n_/n_{+j}). The rcb option controls this across all the
relevant types: "r" for "row", the default, uses outflow counts,
"c" for "col" uses inflow counts, and "b" for "both" calculates the
Euclidean sum of the row and column values.

Measures

The data-based measures depend on the transition table, but do so in number of different ways.
The simple t2s measure works as follows:

t2s: D_ = 2 - p_ - p_, with D_ = 0

The other measures compare all k columns (rows) across rows (columns) i and j, using different quantities:

pct: D_ = sqrt(sum(p_ - p_)^2) where k indexes all destinations (or origins) for states i and j

pearson: PR = (O-E)/sqrt(E); D_ = sqrt(sum(PRik - PRjk)^2)

relres: RR = (O-E)/E; D_ = sqrt(sum(RRik - RRjk)^2)

chisq: D_ = sqrt(sum((p_-p_)^2/n_{+k}))

Author

Brendan Halpin, brendan.halpin@ul.ie

Examples

. use mvad

. sdmksubs, subsmat(flat) id(id) dtype(flat) nstates(6)

. reshape long state, i(id) j(t)

. sdmksubs, state(state) subsmat(sm) id(id) dtype(pct)

. sdmksubs, state(state) subsmat(sml5) id(id) dtype(pct) lag(5)

. corrsqm sm sml5, nodiag

. reshape wide

. oma state*, subs(flat) indel(0.5) pwd(distf) len(72)

. oma state*, subs(sm) indel(0.5) pwd(dist0) len(72)

. oma state*, subs(sml5) indel(0.5) pwd(dist5) len(72)

. corrsqm dist0 dist5, nodiag

See Also

sadi

oma

corrsqm

reshape