Researchers have found that pairing forward and backward contextual representations in a bidirectional LSTM does not improve performance for dependency relation-type classification. In fact, it consistently underperforms same-direction pairing, with the penalty increasing as the distance between tokens grows. The authors used various diagnostic techniques to investigate why this is the case, including freezing the trunk of the model and training only fresh heads, which ruled out training-co-adaptation as a primary cause. They found that there is partial representational redundancy between forward and backward representations, but neither alone nor combined can explain the full gap in performance. --- Why it matters: This matters to researchers working on bidirectional LSTMs because it challenges the conventional wisdom that combining forward and backward contextual representations improves performance for certain tasks. The findings have implications for how these models are designed and trained, particularly when dealing with long-range dependencies. Source: https://arxiv.org/abs/2608.20647