screen scraping - Logging into website with multiple pages using Python (urllib2 and cookielib) -


i writing script retrieve transaction information bank's home banking website use in personal mobile application.

the website laid out so:

https:/ /homebanking.purduefed.com/onlinebanking/login.aspx

-> enter username -> submit form ->

https:/ /homebanking.purduefed.com/onlinebanking/aop/password.aspx

-> enter password -> submit form ->

https:/ /homebanking.purduefed.com/onlinebanking/accountsummary.aspx

the problem having since there 2 separate pages make posts, first thought problem session information being lost. use urllib2's httpcookieprocessor store cookies , make , post requests website, , have found isn't issue.

my current code is:

import urllib import urllib2 import cookielib  loginurl = 'https://homebanking.purduefed.com/onlinebanking/login.aspx' passwordurl = 'https://homebanking.purduefed.com/onlinebanking/aop/password.aspx' accturl = 'https://homebanking.purduefed.com/onlinebanking/accountsummary.aspx'  loginname = 'sample_username' password = 'sample_password'  values = {'loginname' : loginname,       'password' : password}  class myhttpredirecthandler(urllib2.httpredirecthandler):     def http_error_302(self, req, fp, code, msg, headers):         print "cookie manipulation right here"         return urllib2.httpredirecthandler.http_error_302(self, req, fp, code, msg, headers)      http_error_301 = http_error_303 = http_error_307 = http_error_302  login_cred = urllib.urlencode(values)  jar = cookielib.cookiejar() cookieprocessor = urllib2.httpcookieprocessor(jar)  opener = urllib2.build_opener(myhttpredirecthandler, cookieprocessor) urllib2.install_opener(opener) opener.addheaders = [('user-agent', 'mozilla/5.0 (windows; u; windows nt 5.1; de; rv:1.9.1.5) gecko/20091102 firefox/3.5.5')]  opener.addheader = [('referer', loginurl)] response = opener.open(loginurl, login_cred)  reqpage = opener.open(passwordurl)  opener.addheader = [('referer', passwordurl)] response2 = opener.open(passwordurl, login_cred)  reqpage2 = opener.open(accturl)  content = reqpage2.read()

currently, script makes passwordurl page, username posted correctly, when post made passwordurl page, instead of going accturl, redirected login page (the redirect location if accturl opened without proper or lack of credentials).

any thoughts or comments on how move forward appreciated @ point!


Comments

Post a Comment

Popular posts from this blog

php - Wordpress website dashboard page or post editor content is not showing but front end data is showing properly -

Image
recently have uploaded wordpress site onto server. have uploaded wordpress site in sub-folder under root folder ../root(domain root folder)/wordpress (where host wordpress folder). now problem content data in page or post in dashboard of wordpress admin panel not showing - see image. but data showing in front view - see image. if not browser cache issue , not issue happened tinymce editor during upgrade; try add config file: define('concatenate_scripts', false);
Read more

How to get the ip address of VM and use it to configure SSH connection dynamically in Ansible -

i trying ip address of vm , trying use establish ssh vm. following script: --- - hosts: localhost become: yes connection: local gather_facts: false serial: 1 vars_files: - createvmvars.yml tasks: - name: gathering vm info. vsphere_guest: vcenter_hostname: "{{vcenter_hostname}}" username: "{{vcenter_username}}" password: "{{vcenter_password}}" guest: "{{guest_name}}" vmware_guest_facts: yes register: var - debug: msg="{{var.ansible_facts.hw_eth0.ipaddresses[0]}}" - name: establishing ssh connection. script: /home/shasha/devops/scripts/ssh_configure.sh "{{var.ansible_facts.hw_eth0.ipaddresses[0]}}" following content of ssh_configure.sh : #!/bin/sh ssh-keygen -t rsa ssh-copy-id $1 but when running playbook, getting following error while executing third task: error! 'var' undefined but debug module(second task) printin
Read more

javascript - Get parameter of GET request -

i make request: createxhrrequest( "get", fileurl, function( err, response ) { if( err ) { alert( "error get!" ); } alert(response); }); and full response: { "status" : "ok", "message" : "jvberi0xljqkjdduxdgkmy how can jvberi0xlj ? i try response.message , response["message"]. nothing works. just simple access this: var json = json.parse(response); json.message
Read more