Computer Science Enthusiasts in SIS-SMU

In this post, we would like to present some parameter estimation methods common with discrete probability distribution, which is very popular in text modeling. Then we explain the model of Latent Dirichlet Allocation (LDA) in detail. I. Introduction There are two inference problems in parameter estimation: (1) how to estimate values for a set of distribution parameters theta that can best explain a set of observation data. (2) calculate the probability of a new observation given by previous observation. We introduce Bayes' rule to solve these problem above. Bayes' rule is defined as: and may be called:  We will introduce maximum likelihood, a posteriori and Bayesian estimation, central concepts like conjugate distributions and Bayesian networks to tackle two problems. II. Parameter estimation methods 1. Maximum likelihood estimation Maximum likelihood (ML) tries to find the parameters that maximize the likelihood The common way to obtain the parameter es...

(Last updated: 05/12/2015) 1. Why we need to study neural computation ? - To understand how the brain actually works. - To understand a style of parallel computation inspired by neurons and their adaptive connections - To solve practice problems by using novel learning algorithms inspired by the brain 2. What are neural networks ? A typical cortical neuron has a gross physical structure consisting of    1) a cell body    2) an axon where it sends messages to other neurons    3) a dendritic tree where it receives messages from other neurons - Axon contacts dendritic trees at synapses - Spike generation: enough charge in its dendritic tree, depolarize an axon hillock (part of a cell body), sends a spike out along its axon. (Spike: a wave of depolarization travelling along the axom). Synapses    - contains little vesicles of transmitter chemical (ones implementing positive weights and ones implementing negative weights) ...