Golang http get请求在部分但不是全部URL上中断

Question

Right now I'm fetching urls from indiegogo as part of a side project using the basic get request template found [here][1]. I then translate the byte data into a string using

responseText, err:= ioutil.ReadAll(response.Body)
trueText:= string(responseText)

with appropriate error handling where needed

It works fine for repeated attempts at getting and some other urls of varying length(at least as large as the previous url and some longer than the next).

Strangely, when I attempt to get it breaks and throws a runtime error of

panic: runtime error: index out of range

and exits with a status of 2. I'm curious as to what the issue could be.

I know it isn't indiegogo getting angry about my once a minute requests and cutting my connection because I can request continiously for 20 minutes at with no issue. Give it a bit of downtime and it still completely breaks on

Thanks for the assistance

EDIT, it appears as though it was a malformed bit of html in some of the pages that messed with a loop I was running based on the content that managed to break go in the runtime on only some urls. Thanks for the help

[1]:

Answer 1

There is no error when getting from the url and converting the body to the Go string type. For example,

package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
)

func main() {
    url := "http://www.indiegogo.com/projects/culcharge-smallest-usb-charge-and-data-cable-for-iphone-and-android"
    res, err := http.Get(url)
    if err != nil {
        log.Fatal(err)
    }
    body, err := ioutil.ReadAll(res.Body)
    res.Body.Close()
    if err != nil {
        log.Fatal(err)
    }
    text := string(body)
    fmt.Println(len(body), len(text))
}

Output:

66363 66363

You didn't provide us with a small fragment of code which compiles, runs, and fails in the manner you describe. That leaves us all guessing.