Amino Acid Embeddings

[LivC193]

Hi I was wondering based on Figure 1 from your paper if you identified some optimal amino acid embeddings. Most of the people in the field are using one-hot embeddings which lack a lot of information. The best one that I found is :
https://link.springer.com/article/10.1007/s008940100038 with a 7 dimension embedding for each amino acid.

Based on the scale of your model I assume yours will be much better.

[joshim5]

Hi @LivC182, our model learns the amino acid embeddings from the data. If you want to learn more about this, a good search term is "word embedding." Briefly, we take the one-hot representation of the amino acid and pass it through a learned weight matrix W (dimensions 20 x embed_dim). After pre-training, to recover the embedding corresponding to a given amino acid, we compute one_hot(amino acid) * W. Intuitively, this is retrieves the embed_dim-dimensional column vector at the position encoded by the one-hot. Hope that helps!