Can I get embeddings from this model?
#3
by sebastiansarasti - opened
I want an embedding model for time-series. I'm not sure, if this is possible with this model?
You can look at the last hidden_states for every patch and use that as something like an embedding for every group of 32 values, but you can't use the model as is to generate embeddings for a full timeseries.
I want to add that aggregation of these patches can lead to nice representation of the full time series.
It might also make sense to utilize hidden representations of the intermediate layers.
See our paper in https://openreview.net/forum?id=vhngjDHyB0