Prétraitement des logs¶
Links: notebook, html, PDF, python, slides, GitHub
Traitements de logs générés par un QCM.
from jyquickhelper import add_notebook_menu
add_notebook_menu()
%matplotlib inline
Observations brutes¶
import os
names = [os.path.join("logs", _) for _ in os.listdir("logs") if '.log' in _]
names = names[-1:]
names
['logs\QCMApp.log']
with open(names[0], 'r', encoding="utf-8") as f:
lines = f.readlines()
lines[5:7]
['2018-12-12 17:56:29,989,INFO,[DATA],{"msg":"qcm","session":{"alias":"xavierd"},"client":["167.220.197.38",6274],"game":"simple_french_qcm","qn":"0"}n',
'2018-12-12 17:56:33,130,INFO,[DATA],{"msg":"event","session":{"alias":"xavierd"},"client":["167.220.197.38",6274],"events":["focus:true,game:simple_french_qcm,qn:0"]}n']
Let’s retrieve the expected answers from the QCM.
from mathenjeu.tests import simple_french_qcm
qcm = simple_french_qcm()
expected = qcm.expected_answers()
expected
[['simple_french_qcm-0-a0'],
['simple_french_qcm-1-a2'],
['simple_french_qcm-2-a2'],
['simple_french_qcm-3-a3'],
['simple_french_qcm-4-a2'],
['simple_french_qcm-5-a0', 'simple_french_qcm-5-a2'],
['simple_french_qcm-6-a5'],
['simple_french_qcm-7-a2'],
['simple_french_qcm-8-Thalès']]
import pandas
from mathenjeu.datalog import enumerate_qcmlogdf
dfs = pandas.concat(list(enumerate_qcmlogdf(names, expected)))
dfs.T
| person_id | c241c15008614ea67480 | 32606f02fa0df6aac111 | 5bd96adafa694996027d | 753a507a9205e4c196fa | 8a8c40ad28eb1206efd5 |
|---|---|---|---|---|---|
| simple_french_qcm-0-a0 | 1 | NaN | 1 | 1 | NaN |
| simple_french_qcm-0-b | 1 | NaN | 1 | 1 | 1 |
| simple_french_qcm-0-duration | 0 days 00:00:04.266000 | NaT | 0 days 00:00:06.904000 | 0 days 00:00:16.530000 | 0 days 00:00:02.920000 |
| simple_french_qcm-0-good | 1 | NaN | 1 | 1 | 0 |
| simple_french_qcm-0-nbvisit | 1 | NaN | 1 | 1 | 1 |
| simple_french_qcm-5-a0 | 1 | NaN | NaN | 1 | NaN |
| simple_french_qcm-5-a1 | 1 | NaN | 1 | NaN | NaN |
| simple_french_qcm-5-a2 | 1 | NaN | NaN | NaN | NaN |
| simple_french_qcm-5-b | 1 | NaN | 1 | 1 | 1000 |
| simple_french_qcm-5-duration | 1 days 00:00:00 | NaT | 0 days 00:00:10.153000 | 0 days 00:00:11.740000 | 0 days 00:00:02.247000 |
| simple_french_qcm-5-good | 1 | NaN | 0 | 1 | 0 |
| simple_french_qcm-5-nbvisit | 0.5 | NaN | 1 | 1 | 2 |
| simple_french_qcm-8-ANS | thalès | NaN | Astérix et Cléopâtre | Prout | |
| simple_french_qcm-8-b | 1 | NaN | 1 | 1 | 1 |
| simple_french_qcm-8-duration | 0 days 00:00:04.390000 | NaT | 0 days 00:03:02.506000 | 0 days 00:00:14.483000 | 0 days 00:00:01.422000 |
| simple_french_qcm-8-good | 0 | NaN | 0 | 0 | 0 |
| simple_french_qcm-8-nbvisit | 1 | NaN | 1 | 1 | 1 |
| simple_french_qcm-1-a2 | NaN | 1 | 2 | NaN | NaN |
| simple_french_qcm-1-b | NaN | 1 | 1001 | 1 | NaN |
| simple_french_qcm-1-duration | NaT | 0 days 00:00:03.343000 | 0 | 0 days 00:00:14.010000 | NaT |
| simple_french_qcm-1-good | NaN | 1 | 2 | 0 | NaN |
| simple_french_qcm-1-nbvisit | NaN | 1 | 3 | 1 | NaN |
| simple_french_qcm-2-a2 | NaN | 1 | 1 | 1 | NaN |
| simple_french_qcm-2-b | NaN | 1 | 1 | 1 | NaN |
| simple_french_qcm-2-duration | NaT | 0 days 00:00:06.750000 | 0 days 00:00:31.978000 | 0 days 00:00:28.765000 | NaT |
| simple_french_qcm-2-good | NaN | 1 | 1 | 1 | NaN |
| simple_french_qcm-2-nbvisit | NaN | 1 | 2 | 1 | NaN |
| simple_french_qcm-3-a2 | NaN | 1 | NaN | NaN | 1 |
| simple_french_qcm-3-a3 | NaN | 1 | NaN | NaN | 1 |
| simple_french_qcm-3-b | NaN | 1 | 1 | 1 | 1000 |
| simple_french_qcm-3-duration | NaT | 1 days 00:00:00 | 0 days 00:00:19.246000 | 0 days 00:00:19.492000 | 0 days 00:00:09.323000 |
| simple_french_qcm-3-good | NaN | 1 | 0 | 0 | 1 |
| simple_french_qcm-3-nbvisit | NaN | 0.5 | 1 | 1 | 3 |
| simple_french_qcm-4-a2 | NaN | 1 | 1 | NaN | 1 |
| simple_french_qcm-4-b | NaN | 1 | 1 | 1 | 2 |
| simple_french_qcm-4-duration | NaT | 0 days 00:00:09.688000 | 0 days 00:00:21.230000 | 0 days 00:03:19.593000 | 0 |
| simple_french_qcm-4-good | NaN | 1 | 1 | 0 | 1 |
| simple_french_qcm-4-nbvisit | NaN | 1 | 1 | 1 | 5 |
| simple_french_qcm-6-a3 | NaN | 1 | NaN | NaN | NaN |
| simple_french_qcm-6-b | NaN | 1 | 1 | 1 | 1 |
| simple_french_qcm-6-duration | NaT | 1 days 00:00:00 | 0 days 00:00:20.314000 | 0 days 00:00:21.868000 | 0 days 00:00:00.750000 |
| simple_french_qcm-6-good | NaN | 0 | 0 | 1 | 0 |
| simple_french_qcm-6-nbvisit | NaN | 0.5 | 1 | 1 | 1 |
| simple_french_qcm-7-a2 | NaN | 1 | 1 | NaN | NaN |
| simple_french_qcm-7-b | NaN | 1 | 1 | 1 | 1 |
| simple_french_qcm-7-duration | NaT | 0 days 00:00:06.390000 | 0 days 00:00:17.141000 | 0 days 00:00:20.923000 | 0 days 00:00:01.188000 |
| simple_french_qcm-7-good | NaN | 1 | 1 | 0 | 0 |
| simple_french_qcm-7-nbvisit | NaN | 1 | 1 | 1 | 1 |
| simple_french_qcm-3-a0 | NaN | NaN | 1 | 1 | NaN |
| simple_french_qcm-6-a2 | NaN | NaN | 1 | NaN | NaN |
| simple_french_qcm-1-a1 | NaN | NaN | NaN | 1 | NaN |
| simple_french_qcm-4-a0 | NaN | NaN | NaN | 1 | NaN |
| simple_french_qcm-6-a5 | NaN | NaN | NaN | 1 | NaN |
| simple_french_qcm-7-a0 | NaN | NaN | NaN | 1 | NaN |
| -a1 | NaN | NaN | NaN | NaN | NaN |
| on-a2 | NaN | NaN | NaN | NaN | NaN |
| simple_french_qcm-0-a1 | NaN | NaN | NaN | NaN | 1 |
| simple_french_qcm-4-a3 | NaN | NaN | NaN | NaN | 1 |