HTML-Parser / SGML-Parser

HTML-Parser / SGML-Parser

Post by Zach Denni » Thu, 02 Oct 2003 14:35:43


Ok, silly question.

I am writing a script to determine my router's WAN ip address and then to
email me once an hour in case it changes. Currently I am running a web
server at work that returns a page with the client's ip address. I need to
parse out the info on the page so I can extract the ip address of my router
when my script/program connects.

I am using the html-parser, sgml-parser and formatter ruby libraries
provided from raa and I have made the changes to the regexp regarding image
width and height. So I'm good there.

In my test.rb file I say:
------------------------------------------------
h = Net::HTTP.new('www.zachstestip.com' , 80 )
resp,data = h.get('/index.php' , nil )

w = DumbWriter.new
f = AbstractFormatter.new(w)
p = HTMLParser.new(f)
p.feed(data)
p.close
------------------------------------------------

Here comes the silly part. The function "feed" is inherited by sgml-parser
to html-parser. It passes "data" along to the sgml-parser function
"goahead". It prints everything to stdout or stderr( i dont know, but it
makes it to my screen =), but there is no print, put, etc... etc... call to
send it there!!! I cant for the life of me determine where in the feed or
goahead functions are outputting my parsed results from data! This is damn
silly of me to ask I know, but how is it getting to my CLI?

In the "goahead" function there is a giant while loop. If i place a print or
puts statement at the right before the loop and right after the loop, then
nothing is outputted( except for my explicit print/puts statements).

Am I losing it?

Zach
 
 
 

HTML-Parser / SGML-Parser

Post by Sean O'Del » Thu, 02 Oct 2003 15:02:48


Why not just qualify your IP address with something like >>>>IP<<<< and
then you can regex for it like this:

match = />>>>(.+)<<<</.match(HTML)

match[1] => your IP address

Sean O'Dell

 
 
 

HTML-Parser / SGML-Parser

Post by Ara.T.Howa » Thu, 02 Oct 2003 22:07:39


check out dyndns.org - they have scripts for just about every router that does
this.


one thing i might point out here - i myself have spent hours trying to figure
out weird bugs after naming a variable 'p'. worth a check...


you could also try something like this to track the problem:

alias __p p
alias __print print
alias __puts puts

def p(*args);STDERR.p(caller.join("\n")); __p(*args);end
def print(*args);STDERR.print(caller.join("\n")); __print(*args);end
def puts(*args);STDERR.puts(caller.join("\n")); __puts(*args);end

i'm note sure you'd need all three but... you get the picture.

-a
====================================
| Ara Howard
| NOAA Forecast Systems Laboratory
| Information and Technology Services
| Data Systems Group
| R/FST 325 Broadway
| Boulder, CO 80305-3328
| Email: XXXX@XXXXX.COM
| Phone: 303-497-7238
| Fax: 303-497-7259
| The difference between art and science is that science is what we understand
| well enough to explain to a computer. Art is everything else.
| -- Donald Knuth, "Discover"
| ~ > /bin/sh -c 'for lang in ruby perl; do $lang -e "print \"\x3a\x2d\x29\x0a\""; done'
====================================
 
 
 

HTML-Parser / SGML-Parser

Post by Steven Jen » Thu, 02 Oct 2003 23:01:15

This doesn't answer your questions about Ruby, but most of what you want
exists already.

Look at http://www.yqcomputer.com/ . I've been using them for a year or so.
Every 5 minutes, a Perl daemon (ddclient) on my system wakes up, grabs
the WAN address from my Linksys box, and if it's changed, updates
dyndns. I can ssh into my system at home using the name
'tidal.dyndns.org', even though the address actually belongs to my ISP.
It works great, and it's free.

Steve
 
 
 

HTML-Parser / SGML-Parser

Post by Zach Denni » Fri, 03 Oct 2003 00:26:12

Thank you all for your replies, but I do not want to work around the
solution I would like to work the solution. When attempting to learn a new
language I try to think of small projects that I can do to use different
parts of the language. This is just one of them. I am going to attempt to
figure out what Ara had mentioned. Sometimes I have to reinvent little
wheels so I know I am using the language right when it comes to building
module pieces of big 99^nth polygon shapes with 147 sides.

-Zach
 
 
 

HTML-Parser / SGML-Parser

Post by Ben Giddin » Fri, 03 Oct 2003 01:42:33


I think the HTML parser might be abandoned (RAA says the last update was
2001-07-10 13:35:40 GMT).

You might have better luck using (my) htmltokenizer. It has a really
simple interface, and it might be more what you need:

http://www.yqcomputer.com/

If you really want to use the html-parser, sorry, I can't help you. I
never managed to understand how to work it, which is why I ported the
htmltokenizer.

Ben
 
 
 

HTML-Parser / SGML-Parser

Post by Zach Denni » Fri, 03 Oct 2003 01:45:41

I'll check it out! When you say simple, can I extract data from forms in
html page by chance?

Thanks,

-Zach

-----Original Message-----
From: Ben Giddings [mailto: XXXX@XXXXX.COM ]
Sent: Wednesday, October 01, 2003 12:43 PM
To: ruby-talk ML
Subject: Re: HTML-Parser / SGML-Parser



image

I think the HTML parser might be abandoned (RAA says the last update was
2001-07-10 13:35:40 GMT).

You might have better luck using (my) htmltokenizer. It has a really
simple interface, and it might be more what you need:

http://www.yqcomputer.com/

If you really want to use the html-parser, sorry, I can't help you. I
never managed to understand how to work it, which is why I ported the
htmltokenizer.

Ben
 
 
 

HTML-Parser / SGML-Parser

Post by Ben Giddin » Fri, 03 Oct 2003 02:00:28


You should be able to.

If you say:

while token = tokenizer.getTag('input')
next unless 'ip_addr' == token.attr_hash['name']

puts token.attr_hash['value']
end

I think that will do what you want. I'm not sure if the syntax is perfect
since I'm doing this from memory, but it should be close enough to get you
started.

Ben
 
 
 

HTML-Parser / SGML-Parser

Post by Bernard De » Fri, 03 Oct 2003 04:26:11

> ------------------------------------------------

Through the DumbWriter. Check its implementation in
...\Ruby\lib\ruby\site_ruby\formatter.rb
that's where the "write" statements live.

Often times when you want to parse HTML, it is simpler to use
the (misleadingly named) SGMLParser. Anyway these libraries are
direct ports of python modules, and can only be understood by
checking the documentation of the originals.
See eg: http://www.yqcomputer.com/
And usage examples (in python ;-)
http://www.yqcomputer.com/
http://www.yqcomputer.com/ #t4

Cheers,

Bernard.