Activations Glossary

Browse 1 activations terms defined in plain English, from the cultural dictionary of computing.

1 Activations Terms

SwiGLU

A gated activation function variant used in some transformer feed-forward layers to improve model quality relative to simpler activations.

Related Topics

• Neural Networks (1 terms in common)
• Architecture (1 terms in common)