dongxuanyi3406
dongxuanyi3406
2018-02-21 13:47

去元帅嵌套未知字段

I forked a great project here, and have just been having a mess around with it learning some go. My issue I can't figure out is some things about custom unmarshaling, if you see here you can see this unmarshals the Thing struct which holds a Data interface{} field that is then unmarshalled using the Kind string field. This all works great except the nested situations. So the best thing is an example:

Just say you have Thing struct that's kind is a listing, thus the Thing.Data this is unmarshalled to type listing. Then listing has 3 Children of type link that is held in the Children []Thing field. These Children end up being of type map[string]interface {} which is my issue. How can I have this unmarshall down through nested data? So that the Children.Data is also unmarshalled. Is there a way to do this in go or will I have to write a big loop or recursion algorithm? Please let me know what you think or if I am looking at this situation all wrong. Thank you

UPDATE Here is a json sample

{
    "kind": "Listing",
    "data": {
        "after": null,
        "dist": 10,
        "facets": {},
        "modhash": null,
        "whitelist_status": "all_ads",
        "children": [
            {
                "kind": "t3",
                "data": {
                    "domain": "self.nanocurrency",
                    "approved_at_utc": null,
                    "mod_reason_by": null,
                    "banned_by": null,
                    "num_reports": null,
                    "subreddit_id": "t5_2wlj3",
                    "thumbnail_width": null,
                    "subreddit": "CryptoCurrency",
                    "selftext_html": null,
                    "selftext": "",
                    "likes": null,
                    "suggested_sort": null,
                    "crosspost_parent_list": [
                        {
                            "domain": "self.nanocurrency",
                            "approved_at_utc": null,
                            "mod_reason_by": null,
                            "banned_by": null,
                            "num_reports": null,
                            "subreddit_id": "t5_4br49",
                            "thumbnail_width": null,
                            "subreddit": "nanocurrency",
                            "selftext_html": "<!-- SC_OFF --><div class=\"md\"><p>Seeing lots of comments about the lack of timestamps in the protocol, and just wanted to provide some information on why objectively accurate, global time timestamps are an illusion in decentralized systems. Logical time on the other hand is achievable and is in fact achieved in Nano via the individual block chains and the directed, acyclic graph (DAG). </p>

<p>Generally, in a decentralized, distributed system, only logical time (C happened after B happened after A) can be agreed on globally (e.g. Lamport time: <a href=\"https://amturing.acm.org/p558-lamport.pdf\">https://amturing.acm.org/p558-lamport.pdf</a> or vector clocks). This is what the nano ledger does via chaining blocks, just as any other blockchain. Then it connects those chains via a DAG. Syncing global time to logical time is a hard problem that can either be approached via a consensus approach which is prone to sybil attacks, via a probabilistic approach which is what most blockchains go with - essentially relying on their peers - or a centralized approach like NTP which is a no-go in a decentralized system. If you're interested, check e.g. this paper: <a href=\"http://soft.vub.ac.be/%7Etvcutsem/distsys/clocks.pdf\">http://soft.vub.ac.be/~tvcutsem/distsys/clocks.pdf</a></p>

<p>So what you're running into is the problem that while theoretically all or a relevant subset of nodes within the Nano or Bitcoin or Ethereum network could agree to call a certain time "Monday February 12 2018 20:35:23", this timestamp could still be incorrect vs the outside world.</p>

<p>That's why any accurate global timestamps in software today rely on centralized information and trusted entities. The only way this can change is if machines start setting their clocks on their own accurately (e.g. chip-scale atomic clocks)</p>

<p>Some people think this is bad news for Nano because in traditional blockchain, timestamps are fairly important because they are used in new block validation: <a href=\"http://culubas.blogspot.co.uk/2011/05/timejacking-bitcoin_802.html\">http://culubas.blogspot.co.uk/2011/05/timejacking-bitcoin_802.html</a> . Timestamps are a blessing and a curse at the same time in blockchain: They allow for timing and validation of blocks and therefore regulation of coin mining, but also allow for timestamp double spend attacks. Since Nano does not use mining, it is not susceptible to those attacks.</p>

<p>Even with the relative importance of time and timestamps in Bitcoin, timestamp in Bitcoin are not guaranteed to be corresponding to logical time on the chain. Here's the bitcoin wiki talking about the bitcoin block timestamp: <a href=\"https://en.bitcoin.it/wiki/Block_timestamp\">https://en.bitcoin.it/wiki/Block_timestamp</a> </p>

<blockquote>
<p>A timestamp is accepted as valid if it is greater than the median timestamp of previous 11 blocks, and less than the network-adjusted time + 2 hours. "Network-adjusted time" is the median of the timestamps returned by all nodes connected to you. As a result, block timestamps are not exactly accurate, and they do not even need to be in order. Block times are accurate only to within an hour or two. </p>
</blockquote>

<p>This is probably as good as it gets. Still no ordering guarantees though, and a possibilities for attacks which have been discussed again and again. There are certain guards in Bitcoin code to prevent these from happening. </p>

<p>So if you have somebody lecturing you about how architecturally superior coins would have a accurate and monotonically increasing timestamps, point them here, or here: <a href=\"https://cointelegraph.com/news/timestamp-hacking-debunking-the-myth-of-precision-timestamps\">https://cointelegraph.com/news/timestamp-hacking-debunking-the-myth-of-precision-timestamps</a></p>

<p>Would timestamps be convenient to have on the protocol? Yes, and that would likely be a good reason to include them. Are timestamps accurately depicting global time, and can they be used as evidence in any chain? No they can't.</p>

<p>Why are we seeing inaccurate timestamps in the block explorer? Because these are kept in a db outside of the ledger and have been set January 19. Nothing more to it.</p>

<p>Why do people like to babble about how this is a weakness in Nano? No clue.</p>
</div><!-- SC_ON -->",
                            "selftext": "Seeing lots of comments about the lack of timestamps in the protocol, and just wanted to provide some information on why objectively accurate, global time timestamps are an illusion in decentralized systems. Logical time on the other hand is achievable and is in fact achieved in Nano via the individual block chains and the directed, acyclic graph (DAG). 

Generally, in a decentralized, distributed system, only logical time (C happened after B happened after A) can be agreed on globally (e.g. Lamport time: https://amturing.acm.org/p558-lamport.pdf or vector clocks). This is what the nano ledger does via chaining blocks, just as any other blockchain. Then it connects those chains via a DAG. Syncing global time to logical time is a hard problem that can either be approached via a consensus approach which is prone to sybil attacks, via a probabilistic approach which is what most blockchains go with - essentially relying on their peers - or a centralized approach like NTP which is a no-go in a decentralized system. If you're interested, check e.g. this paper: http://soft.vub.ac.be/~tvcutsem/distsys/clocks.pdf

So what you're running into is the problem that while theoretically all or a relevant subset of nodes within the Nano or Bitcoin or Ethereum network could agree to call a certain time \"Monday February 12 2018 20:35:23\", this timestamp could still be incorrect vs the outside world.

That's why any accurate global timestamps in software today rely on centralized information and trusted entities. The only way this can change is if machines start setting their clocks on their own accurately (e.g. chip-scale atomic clocks)

Some people think this is bad news for Nano because in traditional blockchain, timestamps are fairly important because they are used in new block validation: http://culubas.blogspot.co.uk/2011/05/timejacking-bitcoin_802.html . Timestamps are a blessing and a curse at the same time in blockchain: They allow for timing and validation of blocks and therefore regulation of coin mining, but also allow for timestamp double spend attacks. Since Nano does not use mining, it is not susceptible to those attacks.

Even with the relative importance of time and timestamps in Bitcoin, timestamp in Bitcoin are not guaranteed to be corresponding to logical time on the chain. Here's the bitcoin wiki talking about the bitcoin block timestamp: https://en.bitcoin.it/wiki/Block_timestamp 

> A timestamp is accepted as valid if it is greater than the median timestamp of previous 11 blocks, and less than the network-adjusted time + 2 hours. \"Network-adjusted time\" is the median of the timestamps returned by all nodes connected to you. As a result, block timestamps are not exactly accurate, and they do not even need to be in order. Block times are accurate only to within an hour or two. 

This is probably as good as it gets. Still no ordering guarantees though, and a possibilities for attacks which have been discussed again and again. There are certain guards in Bitcoin code to prevent these from happening. 

So if you have somebody lecturing you about how architecturally superior coins would have a accurate and monotonically increasing timestamps, point them here, or here: https://cointelegraph.com/news/timestamp-hacking-debunking-the-myth-of-precision-timestamps

Would timestamps be convenient to have on the protocol? Yes, and that would likely be a good reason to include them. Are timestamps accurately depicting global time, and can they be used as evidence in any chain? No they can't.

Why are we seeing inaccurate timestamps in the block explorer? Because these are kept in a db outside of the ledger and have been set January 19. Nothing more to it.

Why do people like to babble about how this is a weakness in Nano? No clue.",
                            "likes": null,
                            "suggested_sort": null,
                            "user_reports": [],
                            "secure_media": null,
                            "is_reddit_media_domain": false,
                            "link_flair_text": null,
                            "id": "7ww6bm",
                            "banned_at_utc": null,
                            "mod_reason_title": null,
                            "view_count": null,
                            "archived": false,
                            "clicked": false,
                            "media_embed": {},
                            "report_reasons": null,
                            "author": "ohlookaballoon",
                            "num_crossposts": 2,
                            "saved": false,
                            "mod_reports": [],
                            "can_mod_post": false,
                            "is_crosspostable": true,
                            "pinned": false,
                            "score": 24,
                            "approved_by": null,
                            "over_18": false,
                            "hidden": false,
                            "thumbnail": "self",
                            "edited": false,
                            "link_flair_css_class": null,
                            "author_flair_css_class": null,
                            "contest_mode": false,
                            "gilded": 0,
                            "downs": 0,
                            "brand_safe": false,
                            "secure_media_embed": {},
                            "removal_reason": null,
                            "author_flair_text": null,
                            "stickied": false,
                            "can_gild": true,
                            "thumbnail_height": null,
                            "parent_whitelist_status": null,
                            "name": "t3_7ww6bm",
                            "spoiler": false,
                            "permalink": "/r/nanocurrency/comments/7ww6bm/why_global_time_objectively_accurate_timestamps/",
                            "subreddit_type": "public",
                            "locked": false,
                            "hide_score": false,
                            "created": 1518417258,
                            "url": "https://www.reddit.com/r/nanocurrency/comments/7ww6bm/why_global_time_objectively_accurate_timestamps/",
                            "whitelist_status": null,
                            "quarantine": false,
                            "title": "Why global time, objectively accurate timestamps are not achievable in decentralized systems",
                            "created_utc": 1518388458,
                            "subreddit_name_prefixed": "r/nanocurrency",
                            "ups": 24,
                            "media": null,
                            "num_comments": 18,
                            "is_self": true,
                            "visited": false,
                            "mod_note": null,
                            "is_video": false,
                            "distinguished": null
                        }
                    ],
                    "user_reports": [],
                    "secure_media": null,
                    "is_reddit_media_domain": false,
                    "link_flair_text": null,
                    "id": "7x3r45",
                    "banned_at_utc": null,
                    "mod_reason_title": null,
                    "view_count": null,
                    "archived": false,
                    "clicked": false,
                    "media_embed": {},
                    "report_reasons": null,
                    "author": "ohlookaballoon",
                    "num_crossposts": 0,
                    "saved": false,
                    "mod_reports": [],
                    "can_mod_post": false,
                    "is_crosspostable": true,
                    "pinned": false,
                    "score": 2,
                    "approved_by": null,
                    "over_18": false,
                    "hidden": false,
                    "thumbnail": "default",
                    "edited": false,
                    "link_flair_css_class": null,
                    "author_flair_css_class": "New",
                    "contest_mode": false,
                    "gilded": 0,
                    "downs": 0,
                    "brand_safe": true,
                    "secure_media_embed": {},
                    "removal_reason": null,
                    "author_flair_text": "Redditor for 27 days.",
                    "stickied": false,
                    "can_gild": true,
                    "thumbnail_height": null,
                    "parent_whitelist_status": "all_ads",
                    "name": "t3_7x3r45",
                    "crosspost_parent": "t3_7ww6bm",
                    "spoiler": false,
                    "permalink": "/r/CryptoCurrency/comments/7x3r45/why_global_time_objectively_accurate_timestamps/",
                    "subreddit_type": "public",
                    "locked": false,
                    "hide_score": false,
                    "created": 1518496223,
                    "url": "https://np.reddit.com/r/nanocurrency/comments/7ww6bm/why_global_time_objectively_accurate_timestamps/",
                    "whitelist_status": "all_ads",
                    "quarantine": false,
                    "title": "Why global time, objectively accurate timestamps are not achievable in decentralized systems",
                    "created_utc": 1518467423,
                    "subreddit_name_prefixed": "r/CryptoCurrency",
                    "ups": 2,
                    "media": null,
                    "num_comments": 0,
                    "is_self": false,
                    "visited": false,
                    "mod_note": null,
                    "is_video": false,
                    "distinguished": null
                }
            },
  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

2条回答

  • doufu6504 doufu6504 3年前

    This is exactly what json.RawMessage is for (look at the unmarshal example in the docs). Unmarshal the top-level of the JSON Object first, inspect the kind field, then unmarshal the data field:

    type Listing struct {                                           
        WhitelistStatus string  `json:"whitelist_status"`           
        Children        []Thing `json:"children"`                   
    }                                                               
    
    type T3 struct {                                                
        Domain              string `json:"domain"`                  
        CrosspostParentList []struct {                              
                Domain string `json:"domain"`                       
        } `json:"crosspost_parent_list"`                            
    }                                                               
    
    type Thing struct {
        Kind string      `json:"kind"`
        Data interface{} `json:"data"`
    }
    
    func (t *Thing) UnmarshalJSON(b []byte) error {
        var step1 struct {
                Kind string          `json:"kind"`
                Data json.RawMessage `json:"data"` 
        }
    
        if err := json.Unmarshal(b, &step1); err != nil {
                return err
        }
    
        var step2 interface{}
        switch step1.Kind {
        case "Listing":
                step2 = &Listing{}
        case "t3":
                step2 = &T3{}
        default:
                return errors.New("unknown kind: " + step1.Kind) // or simply ignore
        }
    
        if err := json.Unmarshal(b, step2); err != nil {
                return err
        }
    
        t.Kind = step1.Kind
        t.Data = step2
    
        return nil
    }
    

    Try it on the playground: https://play.golang.org/p/giBVT2IWPd-

    点赞 评论 复制链接分享
  • doushenmao9036 doushenmao9036 3年前

    In your case you needs to create a recursive function which will get the nested value of underlying interface{} till last depth. There is no way we can get nested value from interface{} with unknown underlying type. Here is an example of what you can do in your code

    点赞 评论 复制链接分享