[Home] Module Microsoft.FSharp.Compatibility.OCaml.Lexing


Lexing: ML-like lexing support This file maintains rough compatibility for lexbuffers used by some ML laxer generators. The lexbuf carries an associated pair of positions. Beware that only the "cnum" (absolute character number) field is automatically updated as each lexeme is matched. Upon each successful match the prior end position is transferred to be the start position and a new start position is allocated with an updated pos_cnum field.

Type Definitions

TypeDescription
type lexbuf ASCII LexBuffers carrying OCaml-compatible position information. The type "lexbuf" is opaque, but has an internal position information field that can be updated by setting "lexbuf.EndPos", for example if you wish to update the other fields in that position data before or during lexing. You will need to do this if you wish to maintain accurate line-count information. If you do this and wish to maintain strict cross-compiling compatibility with other langauges you may need code to conditionally use lexbuf_set_curr_p when compiling F# code. Note: an abbreviation for LexBuffer<position,byte>
type position Note: an abbreviation for Position
type Position Position information stored for lexing tokens

Values

ValueDescription
val flush_input : lexbuf -> unit
Remove all input, though don't discard the except the current lexeme
val from_binary_reader : #BinaryReader -> lexbuf
Fuel a lexer using the given BinaryReader.
val from_bytearray : byte [] -> lexbuf
Fuel a lexer from an array of bytes
val from_channel : in_channel -> lexbuf
Fuel a lexer using the given in_channel. The bytes are read using Pervasives.input. If the in_channel is a textual channel the bytes are presented to the lexer by decoding the characters using System.Text.Encoding.ASCII.
val from_function : (byte [] -> int -> int) -> lexbuf
Fuel a lexer from function that fills an array of bytes up to the given length, returning the number of bytes filled.
val from_string : string -> lexbuf
Fuel a lexer from a string, converted to ascii using [[System.Text.Encoding.ASCII.GetBytes]]
val from_text_reader : #Encoding -> #TextReader -> lexbuf
Fuel a lexer using the given TextReader or StreamReader. The characters read are decoded to bytes using the given encoding (e.g. System.Text.Encoding.ASCII) and the bytes presented to the lexer. The encoding used to decode the characters is associated with the expectations of the lexer (e.g. a lexer may be constructed to accept only ASCII or pseudo-UTF8 bytes) and will typically be different to the encoding used to decode the file.
val lexeme : lexbuf -> string
Return the matched string
val lexeme_bytes : lexbuf -> byte array
Return the bytes for the matched string
val lexeme_char : lexbuf -> int -> char
Return a character from the matched string, innterpreting the bytes using an ASCII encoding
val lexeme_end : lexbuf -> int
Return absolute positions into the entire stream of characters
val lexeme_end_p : lexbuf -> position
Return the positions stored in the lexbuf for the matched string
val lexeme_start : lexbuf -> int
Return absolute positions into the entire stream of characters
val lexeme_start_p : lexbuf -> position
Return the positions stored in the lexbuf for the matched string
val lexeme_utf8 : lexbuf -> string
Return the matched string interpreting the bytes using the given Unicode text encoding

Deprecated Values

ValueDescription
[<Obsolete
  ("Consider using either Lexing.from_binary_reader or Lexing.from_text_reader with a character decoding appropriate for the range of characters expected by your lexer specification, e.g. the System.Text.ASCII decoding")>]
val from_stream_reader : StreamReader -> lexbuf

Note: Consider using either Lexing.from_binary_reader or Lexing.from_text_reader with a character decoding appropriate for the range of characters expected by your lexer specification, e.g. the System.Text.ASCII decoding

[<Obsolete
  ("Get the EndPos property in the lexbuf directly, e.g. 'lexbuf.EndPos'")>]
val lexbuf_curr_p : lexbuf -> position

Note: Get the EndPos property in the lexbuf directly, e.g. 'lexbuf.EndPos'

same as lexeme_end_p
[<Obsolete
  ("Set the EndPos property in the lexbuf directly, e.g. 'lexbuf.EndPos <- pos'")>]
val lexbuf_set_curr_p : lexbuf -> position -> unit

Note: Set the EndPos property in the lexbuf directly, e.g. 'lexbuf.EndPos <- pos'

[<Obsolete
  ("Set the StartPos property in the lexbuf directly, e.g. 'lexbuf.StartPos <- pos'")>]
val lexbuf_set_start_p : lexbuf -> position -> unit

Note: Set the StartPos property in the lexbuf directly, e.g. 'lexbuf.StartPos <- pos'

See Also

Microsoft.FSharp.Compatibility.OCaml


Documentation for assembly FSharp.Compatibility, version 1.9.4.19, generated using F# version 1.9.4.19