如何在Golang中仅从HTML中提取文本？

Question

To extract text from HTML, I use a fully HTML5-compliant tokenizer and parser, like this

    s := `
<p>Links:</p><ul><li><a href="foo">Foo</a><li>
<a href="/bar/baz">BarBaz</a></ul><span>TEXT <b>I</b> WANT</span>
<script type='text/javascript'>
/* <![CDATA[ */
var post_notif_widget_ajax_obj = {"ajax_url":"http:\/\/site.com\/wp-admin\/admin-ajax.php","nonce":"9b8270e2ef","processing_msg":"Processing..."};
/* ]]> */
</script>`

    domDocTest := html.NewTokenizer(strings.NewReader(s))
    for tokenType := domDocTest.Next(); tokenType != html.ErrorToken; {
        if tokenType != html.TextToken {
            tokenType = domDocTest.Next()
            continue
        }
        TxtContent := strings.TrimSpace(html.UnescapeString(string(domDocTest.Text())))
        if len(TxtContent) > 0 {
            fmt.Printf("%s
", TxtContent)
        }
        tokenType = domDocTest.Next()
    }

but I got this result

Links:
Foo
BarBaz
TEXT
I
WANT
/* <![CDATA[ */
var post_notif_widget_ajax_obj = {"ajax_url":"http:\/\/site.com\/wp-admin\/admin-ajax.php","nonce":"9b8270e2ef","processing_msg":"Processing..."};
/* ]]> */

I don't want CDATA content. Some idea, how to get only the text content?

Answer 1

As indicated by @Eric Pauley, I look at TextTokens & StartTagTokens. Here is my solution

    s := `
<p>Links:</p><ul><li><a href="foo">Foo</a><li>
<a href="/bar/baz">BarBaz</a></ul><span>TEXT <b>I</b> WANT</span>
<script type='text/javascript'>
/* <![CDATA[ */
var post_notif_widget_ajax_obj = {"ajax_url":"http:\/\/site.com\/wp-admin\/admin-ajax.php","nonce":"9b8270e2ef","processing_msg":"Processing..."};
/* ]]> */
</script>`

    domDocTest := html.NewTokenizer(strings.NewReader(s))
    previousStartTokenTest := domDocTest.Token()
loopDomTest:
    for {
        tt := domDocTest.Next()
        switch {
        case tt == html.ErrorToken:
            break loopDomTest // End of the document,  done
        case tt == html.StartTagToken:
            previousStartTokenTest = domDocTest.Token()
        case tt == html.TextToken:
            if previousStartTokenTest.Data == "script" {
                continue
            }
            TxtContent := strings.TrimSpace(html.UnescapeString(string(domDocTest.Text())))
            if len(TxtContent) > 0 {
                fmt.Printf("%s
", TxtContent)
            }
        }
    }