weixin_39719729
weixin_39719729
2020-12-31 03:15

Basic TCP/IP commands: SEND, LISTEN, & REPLY

This is one idea for how to approach #110. It adds SEND, LISTEN, and REPLY to LDPL as basic socket statements. It's not ready to be merged yet, but I wanted to open this to track progress.

Send statement:

SEND <TEXT-EXPR> TO <TEXT-EXPR> AT <NUMBER-EXPR> IN <TEXT-VAR>

This will open a TCP connection to host/port, send the data, read the response into a variable, then close the connection.

You can use it to make basic HTTP requests currently, but people will want persistent connections for stuff like IRC. So the next step is probably adding something like CONNECT TO <host> AT <port> and CLOSE CONNECTION TO <host> AT <port> to be used in conjunction with SEND.

Then if CONNECT has been called, SEND will re-use that connection and keep it open. Otherwise, it'll open and close a new connection.

Receive statement:

LISTEN ON <TEXT-EXPR> AT <NUMBER-EXPR> AND CALL <SUB-PROCEDURE> WITH <TEXT-VAR>

This starts a basic multi-client TCP server that uses select(). It can only read 1024 byte requests, so we'll have to increase that if we want to support stuff like HTTP POST. Otherwise it should work fine. Currently it closes all connections after sending data.

I, uh, wouldn't call this "industrial strength" or anything like that. But once we get it working right, it'll probably be okay for little dev servers.

Right now the request body is put into the <TEXT-VAR> so it can be accessed in the callback, but I'm going to change it to be a MAP. That way you can get information about the specific client that made the request and all that.

Reply statement:

REPLY WITH <TEXT-EXPR>

This can be used in the LISTEN callback. We may want to add a few more callback-specific statements in the future too, for managing the client connection.

Windows

The examples/echo.ldpl program should work on Linux and MacOS. I haven't added Windows support yet, but I am going to work on that next. It should be possible to have things work the same way.

That's it

I'll update this PR when I make more progress. Any thoughts on the syntax, approach, or other things we're gonna need?

该提问来源于开源项目:Lartu/ldpl

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

15条回答

  • weixin_39719729 weixin_39719729 4月前

    I made a little HTTP server that lists the files in a directory and it almost works:

    https://gist.github.com/dvkt/2edc83cf9294c2e575c8767902c6c33c

    image

    Seems to stop working after the first request so still needs some work.

    点赞 评论 复制链接分享
  • weixin_39654465 weixin_39654465 4月前

    Then if CONNECT has been called, SEND will re-use that connection and keep it open. Otherwise, it'll open and close a new connection.

    Great idea! I love it!

    This starts a basic multi-client TCP server that uses select(). It can only read 1024 byte requests, so we'll have to increase that if we want to support stuff like HTTP POST. Otherwise it should work fine. Currently it closes all connections after sending data.

    Did you choose this size for any reason in particular? :thinking: maybe we could let users specify how many bytes they want (maybe that could be an optional part of the statement? LISTEN ON <TEXT-EXPR> AT <NUMBER-EXPR> AND CALL <SUB-PROCEDURE> WITH <TEXT-VAR> BUFFER SIZE <NUM-EXPR> that defaults to 1024 if the last part isn't used).

    Right now the request body is put into the so it can be accessed in the callback, but I'm going to change it to be a MAP. That way you can get information about the specific client that made the request and all that.

    Are you going to store responses by IP or something like that? It would be nice to be able to know which user sent the data inside the sub-procedure called by listen, in order to facilitate the MAP access to the data.

    This can be used in the LISTEN callback. We may want to add a few more callback-specific statements in the future too, for managing the client connection.

    Is this scope specific? What happens if I use it outside the callback sub-procedure? (It's not that I don't like it, I just want to know how this all works)

    The examples/echo.ldpl program should work on Linux and MacOS. I haven't added Windows support yet, but I am going to work on that next. It should be possible to have things work the same way.

    I looooooooooooove this! Tell me if you need testing under Windows or something~

    Very great job, dvkt! I'm impressed, keep on the good work! As always, thank you very much for your amazing contributions!

    点赞 评论 复制链接分享
  • weixin_39654465 weixin_39654465 4月前

    I made a little HTTP server that lists the files in a directory and it almost works

    Aaaaaaaaaaaaaaaaa this is amaziiiiiiiiiinnnnnnnnnnggggggggggggggggggg!!!!!!!! :smile:

    点赞 评论 复制链接分享
  • weixin_39719729 weixin_39719729 4月前

    Great idea! I love it!

    Ok, wonderful! I'll keep trucking on this PR, then :+1:

    Did you choose this size for any reason in particular?

    Let me get back to you on this - I am going to change how this part works. There may not need to be a limit.

    It would be nice to be able to know which user sent the data inside the sub-procedure called by listen, in order to facilitate the MAP access to the data.

    Yes exactly! Then your callback can easily manage multiple clients by ID or IP.

    Is this scope specific? What happens if I use it outside the callback sub-procedure?

    Yes, basically. It won't compile unless it's used inside a sub-procedure (like RETURN), but you are free to use it in any sub-procedure and it'll reply to the correct client. If the reply somehow fails, it'll set ERRORCODE. The checking is limited though, so you won't get a clear idea of why it failed yet.

    Right now I'm just kind of winging the return codes - once this gets a bit further I'll set up a table on the docs site with all the possible socket errors in one place, and add more. There are a small handful really, lots of chances to blow up!

    I looooooooooooove this! Tell me if you need testing under Windows or something~

    Great, thank you very much! This might take a few more days (or more) to get into order, but it will definitely need testing.

    点赞 评论 复制链接分享
  • weixin_39654465 weixin_39654465 4月前

    Yes exactly! Then your callback can easily manage multiple clients by ID or IP.

    How do you plan to implement this? At the moment sub-procedures cannot receive parameters, don't they? Are you going to add that as well or will this be something else?

    Great, thank you very much! This might take a few more days (or more) to get into order, but it will definitely need testing.

    Will be tested! I love this, the approach is so simple! This makes LDPL way more powerful as well, I love that!

    点赞 评论 复制链接分享
  • weixin_39719729 weixin_39719729 4月前

    How do you plan to implement this? At the moment sub-procedures cannot receive parameters, don't they? Are you going to add that as well or will this be something else?

    This is something else - because the server is single threaded using select(), the callbacks are invoked sequentially. So it just re-uses the same global variable and changes the content before invoking the callback each time:

    https://github.com/Lartu/ldpl/pull/111/files#diff-033f788a6a7c2935152d036c30e8fbb2R525

    This should work as long as there's no threading or anything.

    I love this, the approach is so simple! This makes LDPL way more powerful as well, I love that!

    Thanks! I'm sure we'll have to expand it over time but hopefully this works for some people.

    点赞 评论 复制链接分享
  • weixin_39654465 weixin_39654465 4月前

    Ah! Great approach! I'm really expectant for this!

    点赞 评论 复制链接分享
  • weixin_39719729 weixin_39719729 4月前

    Okay, I think I've got all the first pass functionality in place. I converted the request variable that's used in the server callback to a text map with two keys, ip and data. ip is the client's IP and data is the text that was sent to the socket.

    I also added persistent client-side sockets using CONNECT TO <text> AT <number> and CLOSE CONNECTION.

    
    connect to host at port
    while errorcode is equal to 0 do
        display ">> "
        accept msg
        send msg to host at port in response
        display "<< " response
    repeat
    close connection
    

    Each send will re-use the connection that was opened in the CONNECT TO line.

    Right now you can only have one connection open as a client. We could add more in the future and it'll work without threading, but I wanted to start simple. This should work for stuff like basic HTTP clients or telnet clients. When you're running as a server, the server can handle multiple clients connecting at once, though.

    As before, if you call send without calling CONNECT TO it'll open the connection on its own and close it when it's done.

    Here's the updated web server. All it does is display the contents of the directory it was started in, but it works: https://gist.github.com/dvkt/2edc83cf9294c2e575c8767902c6c33c

    And here's a basic HTTP client. If you give it a URL, it'll try to fetch the contents. It doesn't work for HTTPS, only HTTP: https://gist.github.com/dvkt/8effea54633a3955eeb6fed342b6c8c2

    They work with each other, too!

    server:

    
    ./lweb-bin 1111                                                                   
    HTTP listening at http://localhost:1111
    GET / HTTP/1.1
    Host: localhost
    User-Agent: LDPL/3.0.4
    Accept: */*
    Connection: close
    

    client:

    
    $ ./lget-bin localhost:1111                                                      
    HTTP/1.1 200 OK
    Content-Type: text/html; charset=utf-8
    Content-Length: 420
    
    <h3>Directories</h3> ...
    

    It's still got a few bugs I'm working on (big responses aren't working properly in send) but it's getting there.

    Still planning to tackle Windows support and set up an errors table as the next steps.

    点赞 评论 复制链接分享
  • weixin_39654465 weixin_39654465 4月前

    Woah, great! Very neat job, congratulations! I'm very excited for this! :smile:

    点赞 评论 复制链接分享
  • weixin_39654465 weixin_39654465 4月前

    Right now you can only have one connection open as a client. We could add more in the future and it'll work without threading, but I wanted to start simple.

    This intrigues me. How will you tackle that?

    点赞 评论 复制链接分享
  • weixin_39654465 weixin_39654465 4月前

    Are you still alive, dvkt? (?) Is there anything in this pr I could help you with?

    点赞 评论 复制链接分享
  • weixin_39719729 weixin_39719729 4月前

    Sorry yes, have been away a few days but will get back to it soon.

    点赞 评论 复制链接分享
  • weixin_39654465 weixin_39654465 4月前

    Don't apologize, we just wanted to know if this branch was abandoned or not. Don't trying to rush you, haha, sorry!

    点赞 评论 复制链接分享
  • weixin_39719729 weixin_39719729 4月前

    I need to change my approach with SEND - because sockets are stream oriented, there's no way for us to know when the reply has "finished". So we can't really stick a reply in a variable and pretend it's a response, despite best efforts to fake it.

    Many simple protocols like HTTP and Gopher get around this by closing the connection after every response. So we could just keep reading from the socket until the connection is closed and say that's the "response", but if you've opened a socket to something like a telnet server or MUD it won't work and will just hang forever.

    I'll reopen if I get something better working :+1;

    点赞 评论 复制链接分享
  • weixin_39654465 weixin_39654465 4月前

    Ow :disappointed: Sorry to hear that, dvkt, I'll check what you've done and see if something can be saved :(

    点赞 评论 复制链接分享

相关推荐