EnglishRoberta Constructors
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Overloads
| EnglishRoberta(Stream, Stream, Stream) |
Construct tokenizer object to use with the English Robert model. |
| EnglishRoberta(String, String, String) |
Construct tokenizer object to use with the English Robert model. |
EnglishRoberta(Stream, Stream, Stream)
Construct tokenizer object to use with the English Robert model.
public EnglishRoberta (System.IO.Stream vocabularyStream, System.IO.Stream mergeStream, System.IO.Stream highestOccurrenceMappingStream);new Microsoft.ML.Tokenizers.EnglishRoberta : System.IO.Stream * System.IO.Stream * System.IO.Stream -> Microsoft.ML.Tokenizers.EnglishRobertaPublic Sub New (vocabularyStream As Stream, mergeStream As Stream, highestOccurrenceMappingStream As Stream)Parameters
- vocabularyStream
- Stream
The stream of a JSON file containing the dictionary of string keys and their ids.
- mergeStream
- Stream
The stream of a file containing the tokens's pairs list.
- highestOccurrenceMappingStream
- Stream
Remap the original GPT-2 model Ids to high occurrence ranks and values.
Applies to
EnglishRoberta(String, String, String)
Construct tokenizer object to use with the English Robert model.
public EnglishRoberta (string vocabularyPath, string mergePath, string highestOccurrenceMappingPath);new Microsoft.ML.Tokenizers.EnglishRoberta : string * string * string -> Microsoft.ML.Tokenizers.EnglishRobertaPublic Sub New (vocabularyPath As String, mergePath As String, highestOccurrenceMappingPath As String)Parameters
- vocabularyPath
- String
The JSON file path containing the dictionary of string keys and their ids.
- mergePath
- String
The file path containing the tokens's pairs list.
- highestOccurrenceMappingPath
- String
Remap the original GPT-2 model Ids to high occurrence ranks and values.