Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit history

Commits on Apr 27, 2019

• Upate csv test

  jpeacock29

  committed
  0107855

  Copy full SHA for 0107855

Commits on Apr 21, 2019

• Fix bug with selecting next page; consolidate extraction code

  Show description for de370de
  jpeacock29

  committed
  de370de

  Copy full SHA for de370de

Commits on Apr 20, 2019

• Extract all prices and identify unit prices, dropping old price extraction code; generalize URL css selector; improve handling of missing values

  jpeacock29

  committed
  9f28597

  Copy full SHA for 9f28597

Commits on Apr 11, 2019

• Refactor to improve testability of scraping

  jpeacock29

  committed
  b70034c

  Copy full SHA for b70034c

Commits on Apr 2, 2019

• Refactor price extraction code and make price extraction more flexible

  jpeacock29

  committed
  fe4a09c

  Copy full SHA for fe4a09c

Commits on Mar 30, 2019

• Add unit price extraction and fix bug in price extraction

  jpeacock29

  committed
  b596ec6

  Copy full SHA for b596ec6

• Make csv writer more flexible to include price etc; keep record of all HTML pages; add more logging

  jpeacock29

  committed
  bf9450f

  Copy full SHA for bf9450f

• Remove user agent

  jpeacock29

  committed
  b391b4d

  Copy full SHA for b391b4d

• Update CSS selectors for price, title, average rating and number of ratings; add comments; simplify logic

  jpeacock29

  committed
  7662832

  Copy full SHA for 7662832

Commits on Mar 6, 2019

• Fix for the search url change (https://www.amazon.com/s/field-keywords=XXX => https://www.amazon.com/s?k=XXX) + next_page_url selector

  tducret

  committed
  ab21f31

  Copy full SHA for ab21f31

Commits on Jan 18, 2019

• added price

  candidia

  committed
  d0cd60a

  Copy full SHA for d0cd60a

• added price

  candidia

  committed
  e2ccc09

  Copy full SHA for e2ccc09

Commits on Jan 11, 2019

• v0.1.2 : Added image urls for each product

  tducret

  committed
  25ef930

  Copy full SHA for 25ef930

• Updated to handle case where a "data:image/webp" is returned instead of a regular url + transform the url to a high resolution image url

  tducret

  committed
  62f58bb

  Copy full SHA for 62f58bb

• fixed doctests

  candidia

  committed
  31e5684

  Copy full SHA for 31e5684

• Added images

  candidia

  committed
  b92ba75

  Copy full SHA for b92ba75

Commits on Jan 1, 2019

• Modified Dockerfile to use very small base image (Distroless) + updated version number for a pip release (v0.1.1)

  tducret

  committed
  8c5b4b0

  Copy full SHA for 8c5b4b0

• Modified css selector "a['href']" to "a[href]" (it was not working with beautifulsoup 4.7.0 and soupsieve 1.6)

  tducret

  committed
  ac9c727

  Copy full SHA for ac9c727

• Proper handling of 404 pages (0 product returned and no exception) + changed raw.githack.com url

  tducret

  committed
  b9df061

  Copy full SHA for b9df061

Commits on Dec 28, 2018

• small adjustments to improve coverage

  

  Stephen Dolenc

  committed
  4aced6d

  Copy full SHA for 4aced6d

Commits on Dec 26, 2018

• test fix

  

  Stephen Dolenc

  committed
  1a46cac

  Copy full SHA for 1a46cac

Commits on Dec 25, 2018

• conditional

  

  Stephen Dolenc

  committed
  963c07e

  Copy full SHA for 963c07e

Commits on Dec 19, 2018

• v0.0.9 → v0.1.0 + improved README

  tducret

  committed
  13cff25

  Copy full SHA for 13cff25

• Added ASIN to amazon2csv.py and updated tests and README.md

  tducret

  committed
  dffe61e

  Copy full SHA for dffe61e

• added detection of asin

  andreabisello

  committed
  e8b47d3

  Copy full SHA for e8b47d3

Commits on Jul 17, 2018

• When status_code != 200, raise an exception with info

  tducret

  committed
  876d94a

  Copy full SHA for 876d94a

• Add another string to detect an invalid page (The request could not be satisfied)

  tducret

  committed
  6c422ba

  Copy full SHA for 6c422ba

Commits on Jun 23, 2018

• Removed doctest calls in the main()

  tducret

  committed
  ea5225d

  Copy full SHA for ea5225d

• Added coverage + badge

  tducret

  committed
  d8fd6b4

  Copy full SHA for d8fd6b4

• New version with improved SSLError handling

  tducret

  committed
  529fa50

  Copy full SHA for 529fa50

• Handle the SSLError exception to counter the "(Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_read_bytes', 'sslv3 alert handshake failure')],)",),))"

  tducret

  committed
  81022c9

  Copy full SHA for 81022c9

Commits on Jun 21, 2018

• Added doctest

  tducret

  committed
  c272d86

  Copy full SHA for c272d86

Commits on Jun 20, 2018

• New version with improved reliability

  tducret

  committed
  f759f77

  Copy full SHA for f759f77

• Added another css selectors collection for a new Amazon template

  tducret

  committed
  5fa4fd3

  Copy full SHA for 5fa4fd3

Commits on Jun 14, 2018

• New pip version

  tducret

  committed
  a173849

  Copy full SHA for a173849