{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# 2A.eco - Web-Scraping - pokemon\n", "\n", "Il faut r\u00e9cup\u00e9rer automatiquement des images de pok\u00e9mon depuis le site [pokemondb.net](https://pokemondb.net/pokedex/national)."]}, {"cell_type": "markdown", "metadata": {}, "source": ["Pour cet exercice, nous vous demandons d'obtenir 1) les informations personnelles des 721 pokemons sur le site internet [pokemondb.net](http://pokemondb.net/pokedex/national). Les informations que nous aimerions obtenir au final pour les pokemons sont celles contenues dans 4 tableaux :\n", "\n", "- Pok\u00e9dex data\n", "- Training\n", "- Breeding\n", "- Base stats\n", "\n", "Pour exemple : [Pokemon Database](http://pokemondb.net/pokedex/nincada).\n", "\n", "2) Nous aimerions que vous r\u00e9cup\u00e9riez \u00e9galement les images de chacun des pok\u00e9mons et que vous les enregistriez dans un dossier (indice : utilisez les modules request et [shutil](https://docs.python.org/3/library/shutil.html))\n", "_pour cette question ci, il faut que vous cherchiez de vous m\u00eame certains \u00e9l\u00e9ments, tout n'est pas pr\u00e9sent dans le TD_."]}, {"cell_type": "markdown", "metadata": {}, "source": ["## Correction"]}, {"cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": ["import urllib\n", "import bs4\n", "import collections\n", "import pandas as pd\n", "\n", "# pour le site que nous utilisons, le user agent de python 3 n'est pas bien pass\u00e9 :\n", "# on le change donc pour celui de Mozilla\n", "\n", "req = urllib.request.Request('http://pokemondb.net/pokedex/national', \n", " headers={'User-Agent': 'Mozilla/5.0'})\n", "html = urllib.request.urlopen(req).read()\n", "page = bs4.BeautifulSoup(html, \"lxml\")\n", "\n", "# r\u00e9cup\u00e9rer la liste des noms de pok\u00e9mon\n", "\n", "liste_pokemon =[]\n", "for pokemon in page.findAll('span', {'class': 'infocard-lg-img'}) : \n", " pokemon = pokemon.find('a').get('href').replace(\"/pokedex/\",'')\n", " liste_pokemon.append(pokemon)"]}, {"cell_type": "markdown", "metadata": {}, "source": ["## Fonction pour obtenir les caract\u00e9ristiques de pokemons"]}, {"cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["0 bulbasaur\n", "1 ivysaur\n", "2 venusaur\n", "3 charmander\n", "4 charmeleon\n", "5 charizard\n", "6 squirtle\n", "7 wartortle\n", "8 blastoise\n", "9 caterpie\n", "10 metapod\n", "11 butterfree\n", "12 weedle\n", "13 kakuna\n", "14 beedrill\n", "15 pidgey\n", "16 pidgeotto\n", "17 pidgeot\n", "18 rattata\n", "19 raticate\n", "20 spearow\n", "21 fearow\n"]}, {"data": {"text/html": ["
\n", " | National \u2116 | \n", "name | \n", "Type | \n", "Species | \n", "Height | \n", "Weight | \n", "Abilities | \n", "Local \u2116 | \n", "EV yield | \n", "Catch rate | \n", "... | \n", "Growth Rate | \n", "Egg Groups | \n", "Gender | \n", "Egg cycles | \n", "HP | \n", "Attack | \n", "Defense | \n", "Sp. Atk | \n", "Sp. Def | \n", "Speed | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "001 | \n", "bulbasaur | \n", "Grass Poison | \n", "Seed Pok\u00e9mon | \n", "0.7\u00a0m (2\u203204\u2033) | \n", "6.9\u00a0kg (15.2\u00a0lbs) | \n", "1. OvergrowChlorophyll (hidden ability) | \n", "001 (Red/Blue/Yellow)226 (Gold/Silver/Crystal)... | \n", "1 Special Attack | \n", "45 (5.9% with Pok\u00e9Ball, full HP) | \n", "... | \n", "Medium Slow | \n", "Grass, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884\u20135,140 steps) | \n", "45 | \n", "49 | \n", "49 | \n", "65 | \n", "65 | \n", "45 | \n", "
1 | \n", "002 | \n", "ivysaur | \n", "Grass Poison | \n", "Seed Pok\u00e9mon | \n", "1.0\u00a0m (3\u203203\u2033) | \n", "13.0\u00a0kg (28.7\u00a0lbs) | \n", "1. OvergrowChlorophyll (hidden ability) | \n", "002 (Red/Blue/Yellow)227 (Gold/Silver/Crystal)... | \n", "1 Special Attack, 1 Special Defense | \n", "45 (5.9% with Pok\u00e9Ball, full HP) | \n", "... | \n", "Medium Slow | \n", "Grass, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884\u20135,140 steps) | \n", "60 | \n", "62 | \n", "63 | \n", "80 | \n", "80 | \n", "60 | \n", "
2 | \n", "003 | \n", "venusaur | \n", "Grass Poison | \n", "Seed Pok\u00e9mon | \n", "2.0\u00a0m (6\u203207\u2033) | \n", "100.0\u00a0kg (220.5\u00a0lbs) | \n", "1. OvergrowChlorophyll (hidden ability) | \n", "003 (Red/Blue/Yellow)228 (Gold/Silver/Crystal)... | \n", "2 Special Attack, 1 Special Defense | \n", "45 (5.9% with Pok\u00e9Ball, full HP) | \n", "... | \n", "Medium Slow | \n", "Grass, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884\u20135,140 steps) | \n", "80 | \n", "82 | \n", "83 | \n", "100 | \n", "100 | \n", "80 | \n", "
3 | \n", "004 | \n", "charmander | \n", "Fire | \n", "Lizard Pok\u00e9mon | \n", "0.6\u00a0m (2\u203200\u2033) | \n", "8.5\u00a0kg (18.7\u00a0lbs) | \n", "1. BlazeSolar Power (hidden ability) | \n", "004 (Red/Blue/Yellow)229 (Gold/Silver/Crystal)... | \n", "1 Speed | \n", "45 (5.9% with Pok\u00e9Ball, full HP) | \n", "... | \n", "Medium Slow | \n", "Dragon, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884\u20135,140 steps) | \n", "39 | \n", "52 | \n", "43 | \n", "60 | \n", "50 | \n", "65 | \n", "
4 | \n", "005 | \n", "charmeleon | \n", "Fire | \n", "Flame Pok\u00e9mon | \n", "1.1\u00a0m (3\u203207\u2033) | \n", "19.0\u00a0kg (41.9\u00a0lbs) | \n", "1. BlazeSolar Power (hidden ability) | \n", "005 (Red/Blue/Yellow)230 (Gold/Silver/Crystal)... | \n", "1 Special Attack, 1 Speed | \n", "45 (5.9% with Pok\u00e9Ball, full HP) | \n", "... | \n", "Medium Slow | \n", "Dragon, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884\u20135,140 steps) | \n", "58 | \n", "64 | \n", "58 | \n", "80 | \n", "65 | \n", "80 | \n", "
5 rows \u00d7 22 columns
\n", "