package utf8
Import Path
unicode/utf8 (on go.dev)
Dependency Relation
imports 0 packages, and imported by 26 packages
Involved Source Files
Package utf8 implements functions and constants to support text encoded in
UTF-8. It includes functions to translate between runes and UTF-8 byte sequences.
See https://en.wikipedia.org/wiki/UTF-8
Code Examples
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
buf1 := utf8.AppendRune(nil, 0x10000)
buf2 := utf8.AppendRune([]byte("init"), 0x10000)
fmt.Println(string(buf1))
fmt.Println(string(buf2))
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
b := []byte("Hello, 世界")
for len(b) > 0 {
r, size := utf8.DecodeLastRune(b)
fmt.Printf("%c %v\n", r, size)
b = b[:len(b)-size]
}
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
str := "Hello, 世界"
for len(str) > 0 {
r, size := utf8.DecodeLastRuneInString(str)
fmt.Printf("%c %v\n", r, size)
str = str[:len(str)-size]
}
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
b := []byte("Hello, 世界")
for len(b) > 0 {
r, size := utf8.DecodeRune(b)
fmt.Printf("%c %v\n", r, size)
b = b[size:]
}
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
str := "Hello, 世界"
for len(str) > 0 {
r, size := utf8.DecodeRuneInString(str)
fmt.Printf("%c %v\n", r, size)
str = str[size:]
}
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
r := '世'
buf := make([]byte, 3)
n := utf8.EncodeRune(buf, r)
fmt.Println(buf)
fmt.Println(n)
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
runes := []rune{
// Less than 0, out of range.
-1,
// Greater than 0x10FFFF, out of range.
0x110000,
// The Unicode replacement character.
utf8.RuneError,
}
for i, c := range runes {
buf := make([]byte, 3)
size := utf8.EncodeRune(buf, c)
fmt.Printf("%d: %d %[2]s %d\n", i, buf, size)
}
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
buf := []byte{228, 184, 150} // 世
fmt.Println(utf8.FullRune(buf))
fmt.Println(utf8.FullRune(buf[:2]))
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
str := "世"
fmt.Println(utf8.FullRuneInString(str))
fmt.Println(utf8.FullRuneInString(str[:2]))
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
buf := []byte("Hello, 世界")
fmt.Println("bytes =", len(buf))
fmt.Println("runes =", utf8.RuneCount(buf))
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
str := "Hello, 世界"
fmt.Println("bytes =", len(str))
fmt.Println("runes =", utf8.RuneCountInString(str))
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
fmt.Println(utf8.RuneLen('a'))
fmt.Println(utf8.RuneLen('界'))
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
buf := []byte("a界")
fmt.Println(utf8.RuneStart(buf[0]))
fmt.Println(utf8.RuneStart(buf[1]))
fmt.Println(utf8.RuneStart(buf[2]))
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
valid := []byte("Hello, 世界")
invalid := []byte{0xff, 0xfe, 0xfd}
fmt.Println(utf8.Valid(valid))
fmt.Println(utf8.Valid(invalid))
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
valid := 'a'
invalid := rune(0xfffffff)
fmt.Println(utf8.ValidRune(valid))
fmt.Println(utf8.ValidRune(invalid))
}
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
valid := "Hello, 世界"
invalid := string([]byte{0xff, 0xfe, 0xfd})
fmt.Println(utf8.ValidString(valid))
fmt.Println(utf8.ValidString(invalid))
}
Package-Level Type Names (only one, which is unexported)
Package-Level Functions (total 16, in which 15 are exported)
AppendRune appends the UTF-8 encoding of r to the end of p and
returns the extended buffer. If the rune is out of range,
it appends the encoding of RuneError.
DecodeLastRune unpacks the last UTF-8 encoding in p and returns the rune and
its width in bytes. If p is empty it returns (RuneError, 0). Otherwise, if
the encoding is invalid, it returns (RuneError, 1). Both are impossible
results for correct, non-empty UTF-8.
An encoding is invalid if it is incorrect UTF-8, encodes a rune that is
out of range, or is not the shortest possible UTF-8 encoding for the
value. No other validation is performed.
DecodeLastRuneInString is like DecodeLastRune but its input is a string. If
s is empty it returns (RuneError, 0). Otherwise, if the encoding is invalid,
it returns (RuneError, 1). Both are impossible results for correct,
non-empty UTF-8.
An encoding is invalid if it is incorrect UTF-8, encodes a rune that is
out of range, or is not the shortest possible UTF-8 encoding for the
value. No other validation is performed.
DecodeRune unpacks the first UTF-8 encoding in p and returns the rune and
its width in bytes. If p is empty it returns (RuneError, 0). Otherwise, if
the encoding is invalid, it returns (RuneError, 1). Both are impossible
results for correct, non-empty UTF-8.
An encoding is invalid if it is incorrect UTF-8, encodes a rune that is
out of range, or is not the shortest possible UTF-8 encoding for the
value. No other validation is performed.
DecodeRuneInString is like DecodeRune but its input is a string. If s is
empty it returns (RuneError, 0). Otherwise, if the encoding is invalid, it
returns (RuneError, 1). Both are impossible results for correct, non-empty
UTF-8.
An encoding is invalid if it is incorrect UTF-8, encodes a rune that is
out of range, or is not the shortest possible UTF-8 encoding for the
value. No other validation is performed.
EncodeRune writes into p (which must be large enough) the UTF-8 encoding of the rune.
If the rune is out of range, it writes the encoding of RuneError.
It returns the number of bytes written.
FullRune reports whether the bytes in p begin with a full UTF-8 encoding of a rune.
An invalid encoding is considered a full Rune since it will convert as a width-1 error rune.
FullRuneInString is like FullRune but its input is a string.
RuneCount returns the number of runes in p. Erroneous and short
encodings are treated as single runes of width 1 byte.
RuneCountInString is like RuneCount but its input is a string.
RuneLen returns the number of bytes required to encode the rune.
It returns -1 if the rune is not a valid value to encode in UTF-8.
RuneStart reports whether the byte could be the first byte of an encoded,
possibly invalid rune. Second and subsequent bytes always have the top two
bits set to 10.
Valid reports whether p consists entirely of valid UTF-8-encoded runes.
ValidRune reports whether r can be legally encoded as UTF-8.
Code points that are out of range or a surrogate half are illegal.
ValidString reports whether s consists entirely of valid UTF-8-encoded runes.
Package-Level Variables (total 2, neither is exported)
Package-Level Constants (total 30, in which 4 are exported)
Numbers fundamental to the encoding.
Numbers fundamental to the encoding.
Numbers fundamental to the encoding.
Numbers fundamental to the encoding.
 |
The pages are generated with Golds v0.6.7. (GOOS=linux GOARCH=amd64) Golds is a Go 101 project developed by Tapir Liu. PR and bug reports are welcome and can be submitted to the issue list. Please follow @Go100and1 (reachable from the left QR code) to get the latest news of Golds. |