Here's some simplistic debugging advice: if your browser sees one thing at a given URL, but your LWP program sees another, first try just turning on cookie support, with an empty cookie jar. If that fails, have it read in your browser's cookie file. And if that fails, it's time to start wondering what means the remote site is using for distinguishing your LWP program's requests from your browser's requests.
Every kind of browser sends different HTTP headers besides the very minimal headers that LWP::UserAgent typically sends. For example, whereas an LWP::UserAgent browser by default sends this header line:
Netscape 4.76 sends a header line like this:
User-Agent: Mozilla/4.76 [en] (Win98; U)
And also sends these header fields that an LWP::UserAgent browser doesn't send normally at all:
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */* Accept-Charset: iso-8859-1,*,utf-8 Accept-Encoding: gzip Accept-Language: en-US
(That's assuming you've set your language preferences to U.S.
English). That's on top of any
keep-alive headers that may be sent, if the
browser or any intervening firewall supports that feature (
keep-alive) of HTTP.
Opera 5.12 is not much different:
User-Agent: Opera/5.12 (Windows 98; U) [en] Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Language: en Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
But a recent version of Netscape ...