Rewrote login, fixed timing bug

Update README.md

.github/workflows/pylint.yml

@@ -0,0 +1,25 @@
+name: Pylint
+
+on: [push]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v3
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install pylint
+        pip install .
+    - name: Analyzing the code with pylint
+      run: |
+        pylint temmies
+

.github/workflows/python-publish.yml

@@ -0,0 +1,58 @@
+name: Upload Python Package
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  contents: read
+
+jobs:
+  release-build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.x"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install setuptools wheel twine
+
+      - name: Build release distributions
+        run: |
+          python setup.py bdist_wheel
+
+      - name: Upload distributions
+        uses: actions/upload-artifact@v4
+        with:
+          name: release-dists
+          path: dist/
+
+  pypi-publish:
+    runs-on: ubuntu-latest
+    needs:
+      - release-build
+    permissions:
+      id-token: write
+
+    environment:
+      name: pypi
+
+    steps:
+      - name: Retrieve release distributions
+        uses: actions/download-artifact@v4
+        with:
+          name: release-dists
+          path: dist/
+
+      - name: Publish release distributions to PyPI
+        env:
+          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
+        run: |
+          python -m pip install twine
+          twine upload dist/*

.gitignore

@@ -3,7 +3,6 @@ config.py
 tests/
 pathfinding/
 test.py
-setup.py
 
 #Doc env
 .docs_env

README.md

@@ -1,12 +1,12 @@
 <p align="center">
-  <img src="docs/img/rugemmie.gif" />  
+  <img src="docs/img/temmie.png" width= 200px/>  
 </p>
 <p align="center">
-<a href="https://temmies.readthedocs.io/en/latest/"><img alt="Read the Docs" src="https://img.shields.io/readthedocs/temmies"></a>
+<a href="https://temmies.confest.im"><img alt="Read the Docs" src="https://img.shields.io/readthedocs/temmies"></a>
 <img alt="GitHub" src="https://img.shields.io/github/license/Code-For-Groningen/temmies">
 </p>
 
-A python library which interacts with themis. Uses bs4. I'll try to end development on a somewhat working state. 
+A python library which interacts with [Themis](https://themis.housing.rug.nl/). Uses bs4. I'll try to end development on a somewhat working state. 
 
 ## Intended Features
 * [x] Log in  
@@ -15,12 +15,12 @@ A python library which interacts with themis. Uses bs4. I'll try to end developm
 * [x] Submission status
 
 ## Docs
-[here](http://temmies.rtfd.io/).
+- [Click here](http://temmies.confest.im/).
 
 ## Possible continuations
 * Discord bot
-* CLI program
+* [CLI program](https://github.com/Code-For-Groningen/temmies-cli)
 
 ## Thanks to
 * [Glitchcat](https://glitchcat.github.io/themis-api/), cool docs bro.
-* [Mo](https://github.com/Stylo2k), damn bruh you actually work for Themis
+* [Mo](https://github.com/Stylo2k), damn bruh you actually work for Themis

docs/api.md

@@ -1,5 +1,6 @@
 # Classes
 ---
+
 ## `Themis`
 Creates the initial connection to Themis.
 
@@ -7,15 +8,17 @@ Creates the initial connection to Themis.
 ```python
 from temmies.themis import Themis
 
-themis = Themis("s-number", "password")
+themis = Themis("s-number")
 ```
 
+On the first run, you will be prompted for your password. Then, on the next run(s), you will be able to log in automatically, as the password is stored in the system keyring. If you want to delete it [click here](https://www.google.com/search?hl=en&q=delete%20a%20password%20from%20keyring).
+
 ### Methods
 #### `login()`
 Logs in to Themis. Runs automatically when the class is initialized.
 
-#### `get_year(start, end)`
-Returns an instance of a [`Year`](#year)(academic year) between `start` and `end`. 
+#### `get_year(year_path)`
+Returns an instance of a [`Year`](#year) for the academic year specified by `year_path`.
 
 ```python
 year = themis.get_year(2023, 2024)
@@ -27,7 +30,6 @@ Returns a list of `Year` instances corresponding to all years visible to the use
 ```python
 years = themis.all_years()
 ```
-<sub> I don't see why you would need this, but it's here. </sub>
 
 ----
 
@@ -39,13 +41,20 @@ year = themis.get_year(2023, 2024)
 ```
 
 ### Methods
-#### `get_course(name)`
-Returns an instance of a [`Course`](#course) with the name `name`.
+#### `get_course(course_title)`
+Returns an instance of a [`Course`](#course) with the title `course_title`.
 
 ```python
 pf = year.get_course("Programming Fundamentals (for CS)")
 ```
 
+#### `get_course_by_tag(course_tag)`
+Returns an instance of a [`Course`](#course) using the course identifier `course_tag`.
+
+```python
+ai_course = year.get_course_by_tag("adinc-ai")
+```
+
 #### `all_courses()`
 Returns a list of `Course` instances corresponding to all courses visible to the user in a given `Year`.
 
@@ -59,227 +68,120 @@ courses = year.all_courses()
 ### Usage
 ```python
 pf = year.get_course("Programming Fundamentals (for CS)")
-print(pf.info) # <- course info attribute
 assignments = pf.get_groups()
 ```
 
 ### Methods
 #### `get_groups(full=False)`
-Returns a list of `ExerciseGroup` instances corresponding to all exercise groups visible to the user in a given `Course`. Default argument is `full=False`, which will only return the (name, link) of each exercise and folder in the group. If `full=True`, it will traverse the whole course.
-
-You can traverse the course in both cases, although in different ways. 
-
-When you have fully traversed the course, you can access everything via indices and the `exercises` and `folders` attributes of the `ExerciseGroup` instances:
+Returns a list of `ExerciseGroup` or `Group` instances corresponding to all items visible to the user in a given `Course`. The default argument is `full=False`, which will only return the top-level (name, link) of each item. If `full=True`, it will traverse the whole course.
 
 ```python
-  ai_group = ai_course.get_groups(full=True)
-  exercise = ai_group[7].exercises[1] # Week 11 -> Suitcase packing
-  exercise.submit("suitcase.py", silent=False)```
+ai_groups = ai_course.get_groups(full=True)
+exercise = ai_groups[7].exercises[1]
+exercise.submit(["solution.py"], silent=False)
 ```
 
-This is equivalent to the case in which we don't traverse the full course using `get_group` like so:
-
-```python
-ai_group = ai_course.get_group("Week 11")
-exercise = ai_group.get_group("Suitcase packing")
-exercise.submit("suitcase.py", silent=False)
-```
-
-### `get_group(name, full=False)`
-Returns an instance of an `ExerciseGroup` with the name `name`. Default argument is `full=False`, which will only return the (name, link) of each exercise and folder in the group. If `full=True`, it will traverse the whole group.
+#### `get_group(name, full=False)`
+Returns an instance of an `ExerciseGroup` or `Group` with the name `name`. The default argument is `full=False`, which will only return the (name, link) of the group. If `full=True`, it will traverse the whole group.
 
 ```python
 week1 = pf.get_group("Week 1")
 ```
 
-## `ExerciseGroup`
-Setting the `full` flag to `True` will traverse the whole course. 
-
-You can traverse the course in both cases
-* Both folders and exercises are represented as `ExerciseGroup` instances.
-* Folders will have the `am_exercise` attribute set to `False`.
-* Folders can have the `download_files` method called on them.
-* Exercises can have the `submit`, `download_files` and `download_tcs` method called on them.
-
-
-### Example of folder traversal
-Let's say we have a folder structure like this:
-```
-- Course Name
-  - Week 1
-    - Exercise 1
-    - Exercise 2
-      - Part 1
-      - Part 2
-  - Week 2
-    - Exercise 1
-    - Exercise 2
-```
-And we want to get to `Part 2` of `Week 1`'s `Exercise 2`. We would do this:
+#### `create_group(item_data)`
+Creates and returns a `Group` or `ExerciseGroup` instance based on `item_data`.
 
 ```python
-pf = year.get_course("Programming Fundamentals (for CS)")
-assignments = pf.get_groups()
-week1 = assignments[0] # Week 1
-exercise2 = week1.folders[1] # Exercise 2
-part2 = exercise2.exercises[1] # Part 2
-
-# Or, if you dont want to traverse the whole course:
-week1 = pf.get_group("Week 1")
-exercise2 = week1.get_group("Exercise 2")
-part2 = exercise2.get_group("Part 2")
+group = course.create_group(item_data)
 ```
 
+----
+
+## `Group`
+
+Represents an item in Themis, which can be either a folder (non-submittable) or an assignment (submittable).
 
 ### Methods
-#### `download_files(path=".")`
-Downloads all files in the exercise group to a directory `path`. Defaults to the current directory.
+#### `get_items()`
+Returns all items (groups and assignments) under this group.
 
 ```python
-  assignment.download_files()
+items = week1.get_items()
+```
+
+#### `get_item_by_title(title)`
+Returns a single item by its title (case-insensitive).
+
+```python
+item = week1.get_item_by_title("Exercise 2")
+```
+
+#### `get_status(text=False)`
+Retrieves the status of the group. When `text=True`, returns the status as strings. Otherwise, returns submission objects or strings.
+
+```python
+status = group.get_status()
+leading_submission = status["leading"]
+```
+
+#### `download_files(path=".")`
+Downloads all files available for this group to a directory `path`. Defaults to the current directory.
+
+```python
+group.download_files()
 ```
 
 #### `download_tcs(path=".")`
-Downloads all test cases in the exercise group to a directory `path`. Defaults to the current directory.
+Downloads all test cases for this group to a directory `path`. Defaults to the current directory.
 
 ```python
-  assignment.download_tcs()
+group.download_tcs()
 ```
 
-#### get_group(name, full=False)
-This is used when you want to traverse the course dynamically(not recurse through the whole thing). Of course, you can use it even if you've traversed the whole course, but that would overcomplicate things.
+#### `submit(files, judge=True, wait=True, silent=True)`
+Submits the files to the group. Default arguments are `judge=True`, `wait=True`, and `silent=True`.
 
 ```python
-  # Week 1 -> Exercise 2 -> Part 2
-  week1 = pf.get_groups("Week 1")
-  exercise2 = week1.get_group("Exercise 2")
-  part2 = exercise2.get_group("Part 2")
-
-  # This is equivalent to(but faster than):
-  week1 = pf.get_groups("Week 1", full=True)
-  exercise2 = week1[1]
-  part2 = exercise2[1]
+group.submit(["solution.py"], silent=False)
 ```
 
+----
 
+## `ExerciseGroup`
+Represents a submittable exercise. Inherits from `Group`.
+
+### Additional Methods
 #### `submit(files)`
-Submits the files to the exercise group. Default arguments are `judge=True`, `wait=True` and `silent=True`. `judge` will judge the submission instantly, and `wait` will wait for the submission to finish. Turning off `silent` will print the submission status dynamically.
+Submits files to the exercise. Raises an error if the item is not submittable.
 
 ```python
-  suitcase = ai.get_group("Week 11")
-  suitcase[7].exercises[1].submit("suitcase.py", silent=False)
-  
-  # Or
-  ai.get_group("Week 11").get_group("Suitcase packing").submit("suitcase.py", silent=False)
-  
-  >>> 1: ✅
-  >>> 2: ✅
-  >>> 3: ✅
-  >>> 4: ✅
-  >>> 5: ✅
-  >>> 6: ✅
-  >>> 7: ✅
-  >>> 8: ✅
-  >>> 9: ✅
-  >>> 10: ✅
-  
+exercise.submit(["solution.py"])
 ```
 
-#### `get_status(section=None, text=False)`
-Parses the status of the exercise group(from a given section). If `section` is not `None`, it will return the status of the section. Don't set `section` if you don't know what you're doing.
-
-When `text` is set to `True`, it will return the status as a dictionary of strings. Otherwise, it will return a tuple in the form `(dict(str:str), dict(str:Submission))`. Refer to the [Submission](#submission) class for more information.
-
-```python
-    pf = year.get_course("Programming Fundamentals (for CS)")
-    pf_as = pf.get_group("Lab Session 2")
-    
-    # Get exercise
-    exercise = pf_as.get_group("Recurrence")
-    
-    # Get status
-    status = exercise.get_status()
-    print(status)
-
-  >>> (
-  >>> { # Information [0]
-  >>> 'assignment': 'Recurrence'
-  >>> 'group': 'Y.N. Here'
-  >>> 'status': 'passed: Passed all test cases'
-  >>> 'grade': '2.00'
-  >>> 'total': '2'
-  >>> 'output limit': '1'
-  >>> 'passed': '1'
-  >>> 'leading': '/submission/2023-2024/progfun/lab2/recurrence/@submissions/s1234567/s1234567-1'
-  >>> 'best': '/submission/2023-2024/progfun/lab2/recurrence/@submissions/s1234567/s1234567-1'
-  >>> 'latest': '/submission/2023-2024/progfun/lab2/recurrence/@submissions/s1234567/s1234567-1'
-  >>> 'first pass': '/submission/2023-2024/progfun/lab2/recurrence/@submissions/s1234567/s1234567-1'
-  >>> 'last pass': '/submission/2023-2024/progfun/lab2/recurrence/@submissions/s1234567/s1234567-1'
-  >>> 'visible': 'Yes'
-  >>> }
-  >>> { # Submission instances [1]
-  >>> 'leading': <submission.Submission object at 0x774ea7a48cd0>
-  >>> 'best': <submission.Submission object at 0x774ea79af910>
-  >>> 'latest': <submission.Submission object at 0x774eaa7d3c10>
-  >>> 'first_pass': <submission.Submission object at 0x774ea77ee810>
-  >>> 'last_pass': <submission.Submission object at 0x774ea755de10>
-  >>> }
-  >>>)
-
-
-```
-
-#### `get_all_statuses(text=False)
-Does the same as `get_status`, but for all visible status sections.
-
+----
 
 ## `Submission`
-### Usage
-```python
-submission = pf.get_group("Week 1").get_group("Exercise 1").get_group("Part 1").get_status()[1]["leading"] # Week 1 -> Exercise 1 -> Part 1 -> Leading submission
 
-```
+Represents a submission for a specific exercise.
 
 ### Methods
-#### `test_cases()`
-Returns a list of `TestCase` instances corresponding to all test cases in the submission.
+#### `get_test_cases()`
+Returns a dictionary of test cases and their statuses.
 
 ```python
-  submission = pf.get_group("Week 1").get_group("Exercise 1").get_group("Part 1").get_status()[1]["leading"]
-  submission.test_cases()
-  >>> {'1': 'passed', '2': 'passed', '3': 'passed', '4': 'passed', '5': 'passed', '6': 'passed', '7': 'passed', '8': 'passed', '9': 'passed', '10': 'passed'}
-
+test_cases = submission.get_test_cases()
 ```
 
-#### `info()`
-Returns a dictionary of information about the submission.
+#### `get_info()`
+Returns detailed information about the submission.
 
 ```python
-  submission = pf.get_group("Week 1").get_group("Exercise 1").get_group("Part 1").get_status()[1]["leading"]
-  submission.info()
-
-  >>> {
-  >>>'assignment': 'Part 1',
-  >>>'group': 'Y.N. Here',
-  >>>'uploaded_by': 'Y.N. Here s1234567',
-  >>>'created_on': 'Wed Sep 13 2023 12:51:37 GMT+02002023-09-13T10:51:37.338Z',
-  >>>'submitted_on': 'Wed Sep 13 2023 12:51:37 GMT+02002023-09-13T10:51:37.344Z',
-  >>>'status': 'passed: Passed all test cases',
-  >>>'files': [('recurrence.c',
-  >>>'/file/2023-2024/progfun/lab2/recurrence/%40submissions/s1234567/s1234567-1/source/recurrence.c'),
-  >>>('compile.log',
-  >>>'/file/2023-2024/progfun/lab2/recurrence/%40submissions/s1234567/s1234567-1/output/compile.log')],
-  >>>'language': 'c'
-  >>> }
+info = submission.get_info()
 ```
 
-#### `files()`
-Returns a list of files in the form `(name, link)`.
+#### `get_files()`
+Returns a list of uploaded files in the format `(name, URL)`.
 
 ```python
-  submission = pf.get_group("Week 1").get_group("Exercise 1").get_group("Part 1").get_status()[1]["leading"]
-  submission.files()
-
-  >>> [('recurrence.c', '/file/2023-2024/progfun/lab2/recurrence/%40submissions/s1234567/s1234567-1/source/recurrence.c'), ('compile.log', '/file/2023-2024/progfun/lab2/recurrence/%40submissions/s1234567/s1234567-1/output/compile.log')]
+files = submission.get_files()
 ```
-

docs/changelog.md

@@ -0,0 +1,21 @@
+
+## **Changelog**
+
+### **Version 1.1.0**
+
+#### **Documentation**
+- Fixed method signatures to align with actual functionality.
+- Updated `get_status` to properly handle `Submission` instances.
+- Ensured all class and method examples are consistent with the codebase.
+
+#### **Codebase**
+- Prepended `get_` to all methods in `Submission`
+- Created base `Group` from which `Course` and `ExerciseGroup` inherit.
+- Using system keyring to store passwords (Issue #11)
+
+### **Version 1.2.0**
+
+#### **Codebase**
+- Moved all methods related to downloading files (including test cases) to `Group`.
+- Created `get_test_cases` and `get_files` methods in `Group`.
+- We are now using the [API](https://themis.housing.rug.nl/api/navigation/2023-2024) (which mysteriously appeared) to get the year/course structure.

docs/index.md

@@ -1,5 +1,5 @@
 # Temmies!
-<center>![Temmie](img/rugemmie.gif)</center>
+<center>![Temmie](img/temmie.png)</center>
 
 
 ## What is this?
@@ -21,16 +21,16 @@ pip install temmies
 from temmies.themis import Themis
 
 # Log in
-themis = Themis("s-number", "password")
+themis = Themis("s-number") # You will be prompted for your password
 
 # Get a year
-year = themis.get_year(2023, 2024)
+year = themis.get_year("2023-2024")
 
 # Get a course
 course = year.get_course("Programming Fundamentals (for CS)")
 
 # Get an assignment
-assignment = course.get_assignment("Assignment 1")
+assignment = course.get_group("Assignment 1")
 
 # Submit 2 files
 assignment.submit(["among.c", "us.py"])

mkdocs.yml

@@ -3,4 +3,5 @@ nav:
   - Temmies: index.md
   - API Reference: api.md
   - About: about.md
+  - Change Log: changelog.md
 theme: readthedocs

requirements.txt

@@ -0,0 +1,21 @@
+attrs==25.1.0
+beautifulsoup4==4.12.3
+bs4==0.0.2
+certifi==2024.8.30
+cffi==1.17.1
+charset-normalizer==3.4.0
+cryptography==43.0.3
+idna==3.10
+jaraco.classes==3.4.0
+jaraco.context==6.0.1
+jaraco.functools==4.1.0
+jeepney==0.8.0
+keyring==25.5.0
+lxml==5.3.0
+more-itertools==10.5.0
+pycparser==2.22
+requests==2.32.3
+SecretStorage==3.3.3
+selenium==4.28.1
+soupsieve==2.6
+urllib3==2.2.3

setup.py

@@ -0,0 +1,32 @@
+from setuptools import find_packages, setup
+
+with open("README.md", "r") as f:
+    l_description = f.read()
+    
+setup(
+    name="temmies",
+    version="1.2.124",
+    packages=find_packages(),
+    description="A wrapper for the Themis website",
+    long_description=l_description,
+    long_description_content_type="text/markdown",
+    url="https://github.com/Code-For-Groningen/temmies",
+    author="Boyan K.",
+    author_email="boyan@confest.im",
+    license="GPLv3",
+    classifiers=[
+        "Development Status :: 4 - Beta",
+        "Intended Audience :: Developers",
+        "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
+        "Programming Language :: Python :: 3.9",
+    ],
+    install_requires=[
+        "urllib3",
+        "requests",
+        "lxml",
+        "beautifulsoup4",
+        "keyring",
+        "selenium",
+    ],
+    python_requires=">=3.9",
+)

temmies/__init__.py

@@ -1 +1,8 @@
-from .themis import Themis
+"""
+Entry point for the temmies package.
+"""
+import urllib3
+from .themis import Themis
+
+__all__ = ["Themis"]
+urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

temmies/course.py

@@ -1,72 +1,40 @@
 """
-Houses the Course class which is used to represent a course in a year.
+Represents a course.
+A course is a group that contains exercises or other groups.
 """
 
-from bs4 import BeautifulSoup
-from requests import Session
-
+from .group import Group
 from .exercise_group import ExerciseGroup
-from .exceptions.course_unavailable import CourseUnavailable
-from .exceptions.illegal_action import IllegalAction
 
-
-class Course:
+class Course(Group):
     """
-    get_groups: Get all groups in a course. Set full to True to get all subgroups.
-    get_group: Get a group by name. Set full to True to get all subgroups.
+    Represents a course.
     """
 
-    def __init__(self, url: str, name: str, session: Session, parent):
-        self.url = url
-        self.name = name
-        self.__session = session
-        self.__parent = parent
-        self.__request = self.__session.get(self.url)
-        self.__raw = BeautifulSoup(self.__request.text, "lxml")
-
-        self.__course_available(self.__session.get(self.url))
+    def __init__(self, session, course_path: str, title: str, parent):
+        super().__init__(session, course_path, title, parent)
+        self.course_path = course_path  # e.g., '/2023-2024/adinc-ai'
 
     def __str__(self):
-        return f"Course {self.name} in year {self.__parent.year}"
+        return f"Course({self.title})"
 
-    def __course_available(self, r):
-        # Check if we got an error
-        # print(self.url)
-        if "Something went wrong" in r.text:
-            raise CourseUnavailable(
-                message="'Something went wrong'. Course most likely not found. "
+    def create_group(self, item_data):
+        """
+        Create a subgroup (Group or ExerciseGroup) based on item data.
+        """
+        if item_data.get("submitable", False):
+            return ExerciseGroup(
+                self.session,
+                item_data["path"],
+                item_data["title"],
+                self,
+                item_data.get("submitable", False),
             )
 
-    def get_groups(self, full: bool = False) -> list[ExerciseGroup]:
-        """
-        get_groups(full: bool = False) -> list[ExerciseGroup]
-        Get all groups in a course. Set full to True to get all subgroups.
-        """
-        section = self.__raw.find("div", class_="ass-children")
-        entries = section.find_all("a", href=True)
-        return [
-            ExerciseGroup(
-                f"https://themis.housing.rug.nl{x['href']}",
-                x,
-                self.__session,
-                full
-            )
-            for x in entries
-        ]
-
-    # BAD: Repeated code!!!!
-    def get_group(self, name: str, full: bool = False) -> ExerciseGroup:
-        """
-        get_group(name:str, full:bool = False) -> ExerciseGroup
-        Get a single group by name. Set full to True to get all subgroups as well.
-        """
-        group = self.__raw.find("a", text=name)
-        if not group:
-            raise IllegalAction(message=f"No such group found: {name}")
-
-        return ExerciseGroup(
-            f"https://themis.housing.rug.nl{group['href']}",
-            group,
-            self.__session,
-            full
+        return Group(
+            self.session,
+            item_data["path"],
+            item_data["title"],
+            self,
+            item_data.get("submitable", False),
         )

temmies/exercise_group.py

@@ -1,347 +1,31 @@
 """
-Houses the ExerciseGroup class.
-Represents a group of exercises or a single exercise.
-
+Represents a submittable exercise.
 """
 
-from json import loads
-from time import sleep
 from bs4 import BeautifulSoup
-from .exceptions.illegal_action import IllegalAction
-from .submission import Submission
+from .group import Group
 
-class ExerciseGroup:
+class ExerciseGroup(Group):
     """
-    Methods:
-    `submit`: submit to an exercise
-    `get_group`: get a group by name
-    `download_tcs`: download test cases
-    `download_files`: download files
-    
-    `find_status`: get status for an exercise by name
-    `get_all_statuses`: get all available statuses(useful for multiple exercises)
-    `get_status(idx=0)`: get the available statuses for the exercise. Set the idx if you want to get a specific submission.
-    Attributes:
-    
-    `am_exercise`: returns bool which tells you if the instance is an exercise
-    `folders`: folders in the folder
-    `exercises`: exercises in the folder
-    `test_cases`: test cases in the exercise(if it is an exercise)
-    `files`: files in the exercise/folder
+    Represents a submittable exercise.
     """
-
-    def __init__(self, url: str, soup:BeautifulSoup, session, full: bool):
-        self.url = url
-        self.name = soup.text
-        self.__prev_raw = soup
-        self.__session = session
-        self.__request = self.__session.get(self.url)
-        self.__raw = BeautifulSoup(self.__request.text, "lxml")
-        self.__full = full
-
-    @property
-    def am_exercise(self) -> bool:
-        return "ass-submitable" in self.__prev_raw["class"]
-
-    # Test cases
-    @property
-    def test_cases(self) -> list[str]:
-        section = self.__raw.find_all("div", class_="subsec round shade")
-        tcs = []
-        for div in section:
-            res = div.find("h4", class_="info")
-            if not res:
-                continue
-
-            if "Test cases" in res.text:
-                for case in div.find_all("div", class_="cfg-line"):
-                    if link := case.find("a"):
-                        tcs.append(link)
-        return tcs
-
-    def download_tcs(self, path=".") -> list[str]:
+    # pylint: disable=too-many-arguments, too-many-positional-arguments
+    def __init__(self, session, path: str, title: str, parent, submitable: bool = True):
+        super().__init__(session, path, title, parent, submitable=submitable)
+        self.submit_url = f"{self.base_url}/api/submit{self.path}"
+        self.__find_name()
+    def __find_name(self):
         """
-        download_tcs(path=".") -> list[str]
-        Downloads every test case available from a given exercise. `path` defaults to '.'.
+        Find the name of the exercise group.
         """
-        if not self.am_exercise:
-            raise IllegalAction(message="You are downloading test cases from a folder.")
+        if self.title == "":
+            response = self.session.get(self.base_url + self.path)
+            soup = BeautifulSoup(response.text, "lxml")
+            title_elements = soup.find_all("a", class_="fill accent large")
+            if title_elements:
+                self.title = title_elements[-1].get_text(strip=True)
+            else:
+                self.title = self.path.split("/")[-1]
 
-        for tc in self.test_cases:
-            url = f"https://themis.housing.rug.nl{tc['href']}"
-
-            print(f"Downloading {tc.text}")
-            # download the files
-            with open(f"{path}/{tc.text}", "wb") as f:
-                f.write(self.__session.get(url).content)
-
-        return self.test_cases
-
-    # Files
-    @property
-    def files(self) -> list[str]:
-        details = self.__raw.find("div", id=lambda x: x and x.startswith("details"))
-
-        cfg_lines = details.find_all("div", class_="cfg-line")
-
-        link_list = []
-
-        for line in cfg_lines:
-            key = line.find("span", class_="cfg-key")
-
-            if key and "Downloads" in key.text.strip():
-                # Extract all links in the cfg-val span
-                links = line.find_all("span", class_="cfg-val")
-                for link in links:
-                    a = link.find_all("a")
-                    for i in a:
-                        link_list.append(i)
-
-        return link_list
-
-    def download_files(self, path=".") -> list[str]:
-        """
-        download_files(path=".") -> list[str]
-        Downloads every file available from a given exercise/folder. `path` defaults to '.'.
-        """
-        for file in self.files:
-            print(f"Downloading file {file.text}")
-            url = f"https://themis.housing.rug.nl{file['href']}"
-            with open(f"{path}/{file.text}", "wb") as f:
-                f.write(self.__session.get(url).content)
-        return self.files
-
-    @property
-    def exercises(self) -> list[str] | list["ExerciseGroup"]:
-        if self.am_exercise:
-            return self
-
-        section = self.__raw.find("div", class_="ass-children")
-        try:
-            submittables = section.find_all("a", class_="ass-submitable")
-        except AttributeError:
-            return []
-
-        if not self.__full:
-            return [(x.text, x["href"]) for x in submittables]
-        return [
-            ExerciseGroup(
-                f"https://themis.housing.rug.nl{x['href']}", x, self.__session, True
-            )
-            for x in submittables
-        ]
-
-    @property
-    def folders(self) -> list[str] | list["ExerciseGroup"]:
-        section = self.__raw.find("div", class_="ass-children")
-        try:
-            folders = section.find_all("a", class_="ass-group")
-        except AttributeError:
-            return []
-
-        if not self.__full:
-            return [(x.text, x["href"]) for x in folders]
-
-        return [
-            ExerciseGroup(
-                f"https://themis.housing.rug.nl{x['href']}", x, self.__session, True
-            )
-            for x in folders
-        ]
-
-    # Get by name
-    def get_group( # <- 🗿
-        self, name: str, full: bool = False, link: str = None
-    ) -> "ExerciseGroup":
-        """
-        get_group(name:str, full:bool=False, link:str=None) -> ExerciseGroup | list[ExerciseGroup]
-        Get a single group by name.
-        Set `full` to True to get all subgroups as well.
-        Set `link` to directly fetch a group.
-        """
-        if link:
-            return ExerciseGroup(link, self.__prev_raw, self.__session, full)
-
-        group = self.__raw.find("a", text=name)
-        if not group:
-            raise IllegalAction(message=f"No such group found: {name}")
-
-        return ExerciseGroup(
-            f"https://themis.housing.rug.nl{group['href']}", group, self.__session, full
-        )
-
-    # Wait for result
-    def __wait_for_result(self, url: str, verbose: bool, __printed: list) -> None:
-        # This waits for result and returns a bundled info package
-        r = self.__session.get(url)
-        soup = BeautifulSoup(r.text, "lxml")
-        return self.__parse_table(soup, url, verbose, __printed)
-
-    # Account for judge
-    def __race_condition(self, url: str, verbose: bool) -> None:
-        self.__session.get(url.replace("submission", "judge"))
-        return self.__wait_for_result(url, verbose, [])
-
-    def __parse_table(
-        self, soup: BeautifulSoup, url: str, verbose: bool, __printed: list
-    ) -> dict:
-        cases = soup.find_all("tr", class_="sub-casetop")
-        fail_pass = {}
-        i = 1
-        for case in cases:
-            name = case.find("td", class_="sub-casename").text
-            status = case.find("td", class_="status-icon")
-
-            if "pending" in status.get("class"):
-                return self.__race_condition(url, verbose)
-
-            # queued status-icon
-            if "queued" in status.get("class"):
-                sleep(1)  # <- 🗿
-                return self.__wait_for_result(url, verbose, __printed)
-
-            statuses = {
-                "Passed": ("✅", True),
-                "Wrong output": ("❌", False),
-                "No status": ("🐛", None),
-                "error": ("🐛", None),
-            }
-
-            # Printing and storing
-            found = False
-            for k, v in statuses.items():
-                if k in status.text:
-                    found = True
-                    if verbose and int(name) not in __printed:
-                        print(f"{name}: {v[0]}")
-                    fail_pass[int(name)] = v[1]
-                    break
-            if not found:
-                fail_pass[int(name)] = None
-                if verbose and int(name) not in __printed:
-                    print(f"{name}: Unrecognized status: {status.text}")
-
-            __printed.append(int(name))
-            i += 1
-        return fail_pass
-
-    # Submit
-    def submit(
-        self, files: list, judge: bool = True, wait: bool = True, silent: bool = True
-    ) -> dict | None:
-        """
-        submit(files:list, judge:bool=True, wait:bool=True, silent:bool=True) -> dict | None
-        Submits given files to given exercise. Returns a dictionary of test cases and their status.
-        Set judge to False to not judge the submission.
-        Set wait to False to not wait for the result.
-        Set silent to False to print the results.
-        """
-        form = self.__raw.find("form")
-        if not form:
-            raise IllegalAction(message="You cannot submit to this assignment.")
-
-        url = "https://themis.housing.rug.nl" + form["action"]
-        file_types = loads(form["data-suffixes"])
-        if isinstance(files, str):
-            temp = []
-            temp.append(files)
-            files = temp
-
-        packaged_files = []
-        data = {}
-        found_type = ""
-        for file in files:
-            for t in file_types:
-                if t in file:
-                    found_type = file_types[t]
-                    break
-            if not found_type:
-                print("WARNING: File type not recognized")
-
-            with open(file, "rb") as f:
-                packaged_files.append((found_type, (file, f.read())))
-
-        data = {"judgenow": "true" if judge else "false", "judgeLanguage": found_type if found_type else "none"}
-
-        if not silent:
-            print(f"Submitting to {self.name}")
-            for file in files:
-                print(f"• {file}")
-        resp = self.__session.post(url, files=packaged_files, data=data)
-
-        if not wait or not judge:
-            return resp.url if "@submissions" in resp.url else None
-
-        return self.__wait_for_result(resp.url, not silent, [])
-    
-    def __status_sections(self) -> list[BeautifulSoup]:
-        r = self.__session.get("https://themis.housing.rug.nl" + self.__raw.find("a", text="Status")["href"])
-        
-        soup = BeautifulSoup(r.text, "html.parser")
-        sections = soup.find_all('section', class_=lambda class_: class_ and 'status' in class_.split())
-        
-        return sections
-
-    def __parse_section(self, section:BeautifulSoup, text) -> dict[str, Submission] | dict[str, str]:
-        # The section has a heading and a body. We only care about the body
-        body = section.find("div", class_="sec-body") # Find the body of the section
-        body = body.find("div", class_="subsec-container") # Find the subsec-container
-        body = body.find("div", class_="cfg-container")
-        
-        # Parse the cfg-container
-        parsed = {}
-        
-        # Submission instances go here
-        submissions = {}
-        
-        cfg_lines = body.find_all("div", class_="cfg-line")
-        for line in cfg_lines:
-            key = line.find("span", class_="cfg-key").text.strip().split("\n")[0].replace(":", "").lower()
-            value = line.find("span", class_="cfg-val").text.strip()
-
-            # If there is a span with class tip in the key, it means that the value is a link to a submission
-            if tip := line.find("span", class_="tip"):
-                value = line.find("a")["href"]
-                if not text:
-                    submissions[key.split("\n")[0].lower().replace(" ", "_")] = Submission(value, self.__session)
-            parsed[key] = value
-        
-        if text:
-            return parsed
-        
-        return (parsed, submissions)
-        
-    # I assume that the user would usually request submissions for an assignment,
-    # so I will add a default parameter to the method.
-    
-    def get_status(self, section:list[BeautifulSoup]=None, text:bool=False) -> dict[str, Submission] | dict[str, str]:
-        """Get the available submissions for the exercise. 
-        Set text to True to get the text representation of the submission."""
-        if not section:
-            section = self.__status_sections() 
-        
-        try:
-            section = section[0] # When looking at a single exercise, there is only one status section
-        except IndexError as exc:
-            raise IllegalAction("Invalid status") from exc
-        
-        return self.__parse_section(section, text)
-    
-    def get_all_statuses(self, text:bool=False) -> list[dict[str, str]] | list[dict[str, Submission]]:
-        """ Parses every visible status section. """
-        
-        # This is useless for singular exercises, but if you want the submissions for multiple exercises, you can use this.
-        statuses = []
-        for section in self.__status_sections():
-            if parse := self.__parse_section(section, text):
-                # Find name of the exercise
-                name = section.find("h3").text.replace("Status: ", "").replace("\n", "").replace("\t", "")
-                statuses.append((name,parse))
-        return statuses
-    
-    def find_status(self, name:str, text:bool=False) -> dict[str, Submission] | dict[str, str] | None:
-        """ Find a status block for an exercise by name. """
-        # Find a section which has h3 with the name
-        for section in self.__status_sections():
-            if section.find("h3").text.replace("Status: ", "").replace("\n", "").replace("\t", "") == name:
-                return self.__parse_section(section, text)
+    def __str__(self):
+        return f"ExerciseGroup({self.title})"

temmies/group.py

@@ -0,0 +1,357 @@
+"""
+Abstract-ish Group class for Themis API.
+"""
+
+import os
+from typing import Optional, Union, Dict
+from json import loads
+from time import sleep
+from bs4 import BeautifulSoup
+from .submission import Submission
+
+
+class Group:
+    """
+    Represents an item in Themis.
+    Can be either a folder (non-submittable) or an assignment (submittable).
+    """
+
+    # pylint: disable=too-many-instance-attributes, too-many-arguments, too-many-positional-arguments
+    def __init__(self, session, path: str, title: str, parent=None, submitable: bool = False):
+        self.session = session
+        self.path = path  # e.g., '/2023-2024/adinc-ai/labs'
+        self.title = title
+        self.parent = parent
+        self.submitable = submitable
+        self.base_url = "https://themis.housing.rug.nl"
+        self.api_url = f"{self.base_url}/api/navigation{self.path}"
+        self.classes = []
+
+        # Adjust URL construction to include '/course' when accessing HTML pages
+        if not self.path.startswith('/course/'):
+            group_url = f"{self.base_url}/course{self.path}"
+        else:
+            group_url = f"{self.base_url}{self.path}"
+
+        # Fetch the page and parse it
+        response = self.session.get(group_url)
+        if response.status_code != 200:
+            raise ConnectionError(f"Failed to retrieve page for '{self.title}'.")
+        self._raw = BeautifulSoup(response.text, "lxml")
+
+    def get_items(self) -> list:
+        """
+        Get all items (groups and assignments) under this group.
+        """
+        section = self._raw.find("div", class_="ass-children")
+        if not section:
+            return []
+
+        entries = section.find_all("a", href=True)
+        items = []
+        for x in entries:
+            href = x['href']
+            name = x.text.strip()
+            classes = x.get('class', [])
+            submitable = "ass-submitable" in classes
+            item = Group(
+                session=self.session,
+                path=href,
+                title=name,
+                parent=self,
+                submitable=submitable
+            )
+            items.append(item)
+        return items
+
+    def get_item_by_title(self, title: str):
+        """
+        Get a single item by its title, case-insensitive.
+        """
+        items = self.get_items()
+        for item in items:
+            if (item.title.lower() == title.lower()) or (item.path.split("/")[-1] == title):
+                return item
+        raise ValueError(f"Item '{title}' not found under {self.title}.")
+
+    def get_status(self, text: bool = False) -> Union[Dict[str, Union[str, 'Submission']], None]:
+        """
+        Get the status of the current group, if available.
+        """
+        status_link = self._raw.find("a", text="Status")
+        if not status_link:
+            raise ValueError(
+                "Status information is not available for this group.")
+
+        status_url = f"{self.base_url}{status_link['href']}"
+        response = self.session.get(status_url)
+        if response.status_code != 200:
+            raise ConnectionError(
+                f"Failed to retrieve status page for '{self.title}'.")
+
+        soup = BeautifulSoup(response.text, "lxml")
+        section = soup.find("div", class_="cfg-container")
+
+        if not section:
+            return None
+
+        return self.__parse_status_section(section, text)
+
+    def __parse_status_section(
+        self, section: BeautifulSoup,
+        text: bool
+    ) -> Dict[str, Union[str, 'Submission']]:
+        """
+        Parse the status section of the group and clean up keys.
+        """
+        key_mapping = {
+            "leading the submission that counts towards the grade": "leading",
+            "best the latest submission with the best result": "best",
+            "latest the most recent submission": "latest",
+            "first pass the first submission that passed": "first_pass",
+            "last pass the last submission to pass before the deadline": "last_pass",
+        }
+
+        parsed = {}
+        cfg_lines = section.find_all("div", class_="cfg-line")
+        for line in cfg_lines:
+            key_element = line.find("span", class_="cfg-key")
+            value_element = line.find("span", class_="cfg-val")
+            if not key_element or not value_element:
+                continue
+
+            # Normalize key
+            raw_key = " ".join(key_element.get_text(
+                separator=" ").strip().replace(":", "").lower().split())
+            # Use mapped key if available
+            key = key_mapping.get(raw_key, raw_key)
+
+            # Process value
+            link = value_element.find("a", href=True)
+            if link and not text:
+                href = link["href"]
+                # Construct full URL
+                if href.startswith("/"):
+                    submission_url = href
+                elif href.startswith("http"):
+                    submission_url = href.replace(
+                        "https://themis.housing.rug.nl", "")
+                else:
+                    print(f"Invalid href '{href}' found in status page.")
+                    continue  # Skip this entry if href is invalid
+
+                # Instantiate Submission with submission_url and session
+                submission = Submission(submission_url, self.session)
+                parsed[key] = submission
+            else:
+                parsed[key] = value_element.get_text(separator=" ").strip()
+
+        return parsed
+
+    def get_test_cases(self) -> list[Dict[str, str]]:
+        """
+        Get all test cases for this assignment.
+        """
+        if not self.submitable:
+            raise ValueError(
+                f"No test cases for non-submittable item '{self.title}'.")
+
+        sections = self._raw.find_all("div", class_="subsec round shade")
+        tcs = []
+        for div in sections:
+            res = div.find("h4", class_="info")
+            if res and "Test cases" in res.text:
+                for case in div.find_all("div", class_="cfg-line"):
+                    link = case.find("a")
+                    if link:
+                        tcs.append({
+                            'title': link.text.strip(),
+                            'path': link['href']
+                        })
+        return tcs
+
+    def download_tcs(self, path=".") -> list[str]:
+        """
+        Download all test cases for this assignment.
+        """
+        test_cases = self.get_test_cases()
+        downloaded = []
+        for tc in test_cases:
+            url = f"{self.base_url}{tc['path']}"
+            print(f"Downloading {tc['title']}")
+            response = self.session.get(url)
+            if response.status_code == 200:
+                tc_filename = os.path.join(path, tc['title'])
+                with open(tc_filename, 'wb') as f:
+                    f.write(response.content)
+                downloaded.append(tc_filename)
+            else:
+                print(f"Failed to download test case '{tc['title']}'")
+        return downloaded
+
+    def get_files(self) -> list[Dict[str, str]]:
+        """
+        Get all downloadable files for this assignment.
+        """
+        details = self._raw.find(
+            "div", id=lambda x: x and x.startswith("details"))
+        if not details:
+            return []
+
+        cfg_lines = details.find_all("div", class_="cfg-line")
+        files = []
+
+        for line in cfg_lines:
+            key = line.find("span", class_="cfg-key")
+            if key and "Downloads" in key.text.strip():
+                vals = line.find_all("span", class_="cfg-val")
+                for val in vals:
+                    links = val.find_all("a")
+                    for link in links:
+                        files.append({
+                            'title': link.text.strip(),
+                            'path': link['href']
+                        })
+        return files
+
+    def download_files(self, path=".") -> list[str]:
+        """
+        Download all files available for this assignment.
+        """
+        files = self.get_files()
+        downloaded = []
+        for file in files:
+            print(f"Downloading file '{file['title']}'")
+            url = f"{self.base_url}{file['path']}"
+            response = self.session.get(url)
+            if response.status_code == 200:
+                file_filename = os.path.join(path, file['title'])
+                with open(file_filename, 'wb') as f:
+                    f.write(response.content)
+                downloaded.append(file_filename)
+            else:
+                print(f"Failed to download file '{file['title']}'")
+        return downloaded
+
+    # pylint: disable=too-many-locals
+    def submit(
+        self,
+        files: list[str],
+        judge: bool = True,
+        wait: bool = True,
+        silent: bool = True
+    ) -> Optional[dict]:
+        """
+        Submit files to this assignment.
+        Returns a dictionary of test case results or None if wait is False.
+        """
+        if not self.submitable:
+            raise ValueError(
+                f"Cannot submit to non-submittable item '{self.title}'.")
+
+        form = self._raw.find("form")
+        if not form:
+            raise ValueError("Submission form not found.")
+
+        url = f"{self.base_url}{form['action']}"
+        file_types = loads(form.get("data-suffixes", "{}"))
+
+        if isinstance(files, str):
+            files = [files]
+
+        packaged_files = []
+        data = {}
+        found_type = ""
+
+        for file in files:
+            for suffix, lang in file_types.items():
+                if file.endswith(suffix):
+                    found_type = lang
+                    break
+            if not found_type:
+                print("WARNING: File type not recognized")
+
+            with open(file, "rb") as f:
+                packaged_files.append((found_type, (file, f.read())))
+
+        data = {
+            "judgenow": "true" if judge else "false",
+            "judgeLanguage": found_type if found_type else "none"
+        }
+
+        if not silent:
+            print(f"Submitting to {self.title}")
+            for file in files:
+                print(f"• {file}")
+
+        resp = self.session.post(url, files=packaged_files, data=data)
+
+        if not wait or not judge:
+            return resp.url if "@submissions" in resp.url else None
+
+        return self.__wait_for_result(resp.url, not silent, [])
+
+    def __wait_for_result(self, url: str, verbose: bool, __printed: list) -> dict:
+        """
+        Wait for the submission result and return the test case results.
+        """
+        r = self.session.get(url)
+        soup = BeautifulSoup(r.text, "lxml")
+        return self.__parse_table(soup, url, verbose, __printed)
+
+    def __parse_table(self, soup: BeautifulSoup, url: str, verbose: bool, __printed: list) -> dict:
+        """
+        Parse the results table from the submission result page.
+        Wait until all queued status-icons disappear before parsing.
+        """
+        cases = soup.find_all("tr", class_="sub-casetop")
+        fail_pass = {}
+        any_queued = False
+
+        for case in cases:
+            name = case.find("td", class_="sub-casename").text.strip()
+            status = case.find("td", class_="status-icon")
+
+            status_classes = status.get("class", [])
+            if "queued" in status_classes:
+                any_queued = True
+                break
+
+            if "pending" in status_classes:
+                sleep(1)
+                return self.__wait_for_result(url, verbose, __printed)
+
+            statuses = {
+                "Passed": ("✅", True),
+                "Wrong output": ("❌", False),
+                "No status": ("🐛", None),
+                "error": ("🐛", None),
+            }
+
+            found = False
+            for key, (symbol, value) in statuses.items():
+                if key.lower() in status.text.lower():
+                    found = True
+                    case_number = int(name)
+                    if verbose and case_number not in __printed:
+                        print(f"Case {case_number}: {symbol}")
+                    fail_pass[case_number] = value
+                    break
+
+            if not found:
+                case_number = int(name)
+                fail_pass[case_number] = None
+                if verbose and case_number not in __printed:
+                    print(f"{case_number}: Unrecognized status: {status.text.strip()}")
+
+            __printed.append(case_number)
+
+        # Polling (fix, use ws)
+        if any_queued:
+            sleep(1)
+            return self.__wait_for_result(url, verbose, __printed)
+
+        return fail_pass
+
+    def __str__(self):
+        return f"Group({self.title}, submitable={self.submitable})"

temmies/submission.py

@@ -1,17 +1,20 @@
+# submission.py
+
 """
-File to define the submission class
+File to define the Submission class
 """
 
+from typing import Optional
 from bs4 import BeautifulSoup
 
 class Submission:
     """
     Submission class
-    
+
     Methods:
-    test_cases: Get a dict of test cases status
-    info: Submission information (in details)
-    files: Get a list of uploaded files(as names)
+    get_test_cases: Get a dict of test cases status
+    get_info: Submission information (in details)
+    get_files: Get a list of uploaded files (as names)
     """
     def __init__(self, url: str, session):
         self.url = "https://themis.housing.rug.nl" + url
@@ -24,23 +27,17 @@ class Submission:
         """Clean text"""
         clean = text.replace("\t", "").replace("\n", "")
         if value:
-            return clean
-        return clean.replace(" ", "_").replace(":", "").lower()
+            return clean.strip()
+        return clean.replace(" ", "_").replace(":", "").lower().strip()
 
-    def test_cases(self) -> dict[str, str]:
+    def get_test_cases(self) -> dict[str, str]:
         """Get a dict of test cases status"""
-        # In the submission page, the test cases are in a div with class "sub-cases subsec round shade"
-        # print(self.__raw.prettify())
         cases = self.__raw.find("div", class_=lambda x: x and "sub-cases" in x.split())
         if not cases:
             return {}
 
-        # The test cases are in a table in a div with class "cfg-container"
         cases = cases.find("div", class_="cfg-container")
         cases = cases.find("table")
-        # For each test case, there is a tr with class sub-casetop, which contains 2 tds:
-        # * a td with class "sub-case name" which is a name
-        # * a td with a variable class, which is the status text
 
         results = {}
         for entry in cases.find_all("tr", class_="sub-casetop"):
@@ -48,37 +45,60 @@ class Submission:
             status = entry.find(
                 "td", class_=lambda x: x and "status-icon" in x.split()
             ).text
-            results[name] = self.__clean(status)
+            results[name.strip()] = self.__clean(status)
 
         return results
 
-    def info(self) -> dict[str, str] | None:
+    def get_info(self) -> Optional[dict[str, str]]:
         """Submission information (in details)"""
-        # in div with class subsec round shade where there is an h4 with class info
-        # The info is in a div with class "cfg-container"
         if self.__info:
             return self.__info
 
         for div in self.__raw.find_all("div", class_="subsec round shade"):
-            if h4 := div.find("h4", class_=lambda x: x and "info" in x.split()):
-                if "Details" in h4.text:
-                    # The information is in divs with class "cfg-line"
-                    # With key in span with class "cfg-key" and value in span with class "cfg-value"
-                    info = div.find("div", class_="cfg-container")
-                    info = info.find_all("div", class_="cfg-line")
-                    return {
-                        self.__clean(
-                            key := line.find("span", class_="cfg-key").text
-                        ): 
-                            self.__clean(line.find("span", class_="cfg-val").text, value=True) if "Files" not in key else 
-                            ([(self.__clean(x.text), x["href"]) for x in line.find("span", class_="cfg-val").find_all("a")])
-                        for line in info
-                    }
+            h4 = div.find("h4", class_=lambda x: x and "info" in x.split())
+            if h4 and "Details" in h4.text:
+                info = div.find("div", class_="cfg-container")
+                info_lines = info.find_all("div", class_="cfg-line")
+                self.__info = {
+                    self.__clean(
+                        key := line.find("span", class_="cfg-key").text
+                    ): (
+                        self.__clean(line.find("span", class_="cfg-val").text, value=True)
+                        if "Files" not in key
+                        else [
+                            (self.__clean(a.text), a["href"])
+                            for a in line.find("span", class_="cfg-val").find_all("a")
+                        ]
+                    )
+                    for line in info_lines
+                }
+                return self.__info
         return None
 
-    def files(self) -> list[str] | None:
+    def get_files(self) -> Optional[list[str]]:
         """Get a list of uploaded files in the format [(name, url)]"""
         if not self.__info:
-            self.__info = self.info()
-        
+            self.__info = self.get_info()
         return self.__info.get("files", None)
+
+    # Deprecated methods
+    def info(self):
+        """
+        Deprecated method. Use get_info instead.
+        """
+        print("This method is deprecated and will be deleted soon. Use get_info instead.")
+        return self.get_info()
+
+    def test_cases(self):
+        """
+        Deprecated method. Use get_test_cases instead.
+        """
+        print("This method is deprecated and will be deleted in soon. Use get_test_cases instead.")
+        return self.get_test_cases()
+
+    def files(self):
+        """
+        Deprecated method. Use get_files instead.
+        """
+        print("This method is deprecated and will be deleted in soon. Use get_files instead.")
+        return self.get_files()

temmies/themis.py

@@ -1,37 +1,75 @@
 """
-Main class for the Themis API
-
+Main class for the Themis API using the new JSON endpoints.
 """
 
-import urllib3
 from requests import Session
-from bs4 import BeautifulSoup
+from selenium import webdriver
+from selenium.webdriver.support.ui import WebDriverWait
+from selenium.webdriver.support import expected_conditions as EC
+from selenium.common.exceptions import TimeoutException
+
+from selenium.webdriver.common.by import By
+from selenium.common.exceptions import NoSuchElementException, StaleElementReferenceException
+from json import dumps
 from .year import Year
-from .exceptions.illegal_action import IllegalAction
-
-
-# Disable warnings
-urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
-
+import getpass
+import keyring
 
 class Themis:
     """
-    login: Login to Themis
-    get_year: Get a year object
-    all_years: Get all years
+    Main class for interacting with Themis.
+    - login: Login to Themis
+    - get_year: Get a year object
+    - all_years: Get all years
     """
 
-    def __init__(self, user: str, passwd: str):
-        self.session = self.login(user, passwd)
-        self.years = []
-        self.url = "https://themis.housing.rug.nl/course/"
+    def __init__(self, cookies: dict = None, user=None):
+        """
+        Initialize Themis object, logging in with the given user.
 
-    def login(self, user: str, passwd: str) -> Session:
+        Args:
+            user (str): Username to login with.
+
+        Attributes:
+            user (str): Username.
+            password (str): Password, retrieved from keyring.
+            base_url (str): Base URL of the Themis website.
+            session (requests.Session): Authenticated session.
         """
-        login(self, user: str, passwd: str) -> Session
-        Login to Themis
-        Set user to your student number and passwd to your password
+        self.base_url = "https://themis.housing.rug.nl"
+        self.session = self._setup_agent()
+        
+        self.user, self.password = None, None
+        
+        # Old login logic
+        if user:
+            self.user = user
+            self.password = self._get_password()
+        
+        # Reusing session logic
+        if not cookies:
+            self.session = self.login(self.session)
+        else:
+            self.session.cookies.update(cookies)
+            if not self.check_session():
+                self.session = self.login(self.session)
+    
+    def _get_password(self) -> str:
         """
+        Retrieve the password from the keyring, prompting the user if not found.
+        """
+        password = keyring.get_password(f"{self.user}-temmies", self.user)
+        if not password:
+            print(f"Password for user '{self.user}' not found in keyring.")
+            password = getpass.getpass(
+                prompt=f"Enter password for {self.user}: ")
+            keyring.set_password(f"{self.user}-temmies", self.user, password)
+            print("Password saved securely in keyring.")
+        return password
+
+    def _setup_agent(self) -> Session:
+
+        session = Session()
 
         user_agent = (
             "Mozilla/5.0 (X11; Linux x86_64) "
@@ -39,52 +77,94 @@ class Themis:
             "Chromium/80.0.3987.160 Chrome/80.0.3987.163 Safari/537.36"
         )
 
-        headers = {"user-agent": user_agent}
+        session.headers.update({"User-Agent": user_agent})
 
-        data = {"user": user, "password": passwd, "null": None}
+        return session
 
-        with Session() as s:
-            url = "https://themis.housing.rug.nl/log/in"
-            r = s.get(url, headers=headers, verify=False)
-            soup = BeautifulSoup(r.text, "lxml")
-
-            # get the csrf token and add it to payload
-            csrf_token = soup.find("input", attrs={"name": "_csrf"})["value"]
-            data["_csrf"] = csrf_token
-            data["sudo"] = user.lower()
-
-            # Login
-            r = s.post(url, data=data, headers=headers)
-
-            # check if login was successful
-            log_out = "Welcome, logged in as" in r.text
-            if not log_out:
-                raise IllegalAction(message=f"Login for user {user} failed")
-
-        return s
-
-    def get_year(self, start: int, end: int) -> Year:
+    def check_session(self) -> bool:
         """
-        get_year(self, start: int, end: int) -> Year
-        Gets a year object
-        Set start to the start year and end to the end year (e.g. 2023-2024)
+        Check if the session is still valid.
         """
-        return Year(self.session, start, end)
+        
+        # look at the /login and find a pre tag
+        login_url = f"{self.base_url}/login"
+        response = self.session.get(login_url)
+        return "pre" in response.text
+    
+        
+    def login(self, session: Session) -> Session:
+        """
+        Login to Themis by spawning a selenium browser
+        """
+        login_url = f"{self.base_url}/login"
+        driver = webdriver.Chrome()
 
-    def all_years(self) -> list[Year]:
+        driver.get(login_url)
+
+        wait = WebDriverWait(driver, 60)
+
+        try:
+            wait.until(EC.url_contains("signon.rug.nl/nidp/saml2/sso"))
+            current_url = driver.current_url
+
+            # If on the sign-on page fill in the credentials
+            if "signon.rug.nl/nidp/saml2/sso" in current_url:
+                user_field = wait.until(EC.presence_of_element_located((By.NAME, "Ecom_User_ID")))
+                pass_field = wait.until(EC.presence_of_element_located((By.NAME, "Ecom_Password")))
+                
+                if self.user and not user_field.get_attribute("value"):
+                    user_field.clear()
+                    user_field.send_keys(self.user)
+                if self.password and not pass_field.get_attribute("value"):
+                    pass_field.clear()
+                    pass_field.send_keys(self.password)
+            
+            # THIS IS LIKELY TO BREAK AT SOME POINT
+            wait.until(EC.text_to_be_present_in_element((By.TAG_NAME, "body"), "Cannot GET"))
+            
+        except TimeoutException:
+            print("Timeout waiting for login/2FA page to load.")
+        except (NoSuchElementException, StaleElementReferenceException) as e:
+            print(f"Encountered an error: {e}")
+        finally:
+            # security
+            self.password = "I-HAVE-BEEN-REMOVED"
+            cookies = driver.get_cookies()
+            driver.quit()
+
+        # Add all cookies to the session.
+        for cookie in cookies:
+            session.cookies.set(name=cookie["name"], value=cookie["value"])
+
+        return session
+
+    def get_session_cookies(self):
         """
-        get_years(self, start: int, end: int) -> list[Year]
-        Gets all visible years
+        Get the session cookies in json
         """
-        # All of them are in a big ul at the beginning of the page
-        r = self.session.get(self.url)
-        soup = BeautifulSoup(r.text, "lxml")
-        ul = soup.find("ul", class_="round")
-        lis = ul.find_all("li", class_="large")
+        return dumps(self.session.cookies.get_dict())
+
+    def get_year(self, start_year: int = None, end_year: int = None) -> Year:
+        """
+        Gets a Year object using the year path (e.g., 2023, 2024).
+        """
+        year_path = f"{start_year}-{end_year}"
+
+        return Year(self.session, year_path)
+
+    def all_years(self) -> list:
+        """
+        Gets all visible years as Year objects.
+        """
+        navigation_url = f"{self.base_url}/api/navigation/"
+        response = self.session.get(navigation_url)
+        if response.status_code != 200:
+            raise ConnectionError("Failed to retrieve years from Themis API.")
+
+        years_data = response.json()
         years = []
-        for li in lis:
-            # format: 2019-2020
-            year = li.a.text.split("-")
-            years.append(Year(self.session, int(year[0]), int(year[1])))
-
-        return years  # Return a list of year objects
+        for year_info in years_data:
+            if year_info.get("visible", False):
+                year_path = year_info["path"].strip("/")
+                years.append(Year(self.session, year_path))
+        return years

temmies/year.py

@@ -1,66 +1,75 @@
 """
-Class which represents an academic year.
+This module defines the Year class for managing academic year courses.
 """
 
 from bs4 import BeautifulSoup
-from requests import Session
-
 from .course import Course
-from .exceptions.course_unavailable import CourseUnavailable
 
 
-# Works
 class Year:
     """
-    all_courses: Get all visible courses in a year
-    get_course: Get a course by name
+    Represents an academic year.
     """
+    def __init__(self, session, year_path: str):
+        self.session = session
+        self.year_path = year_path  # e.g., '2023-2024'
+        self.base_url = "https://themis.housing.rug.nl"
+        self.api_url = f"{self.base_url}/api/navigation/{self.year_path}"
 
-    def __init__(self, session: Session, start_year: int, end_year: int):
-        self.start = start_year
-        self.year = end_year
-        self.url = f"https://themis.housing.rug.nl/course/{self.start}-{self.year}"
-        self.__session = session
+    def all_courses(self) -> list:
+        """
+        Gets all visible courses in this year.
+        """
+        response = self.session.get(self.api_url)
+        if response.status_code != 200:
+            raise ConnectionError(f"Failed to retrieve courses for {self.year_path}.")
 
-    # Method to get the courses of the year
-    def all_courses(self, errors: bool = True) -> list[Course]:
-        """
-        all_courses(self, errors: bool = False) -> list[Course]
-        Gets all visible courses in a year.
-        Set errors to False to not raise an error when a course is unavailable.
-        """
-        r = self.__session.get(self.url)
-        soup = BeautifulSoup(r.text, "lxml")
-        lis = soup.find_all("li", class_="large")
+        courses_data = response.json()
         courses = []
-        for li in lis:
-            try:
-                suffix = li.a["href"].replace(f"course/{self.start}-{self.year}", "")
-                courses.append(
-                    Course(self.url + suffix, li.a.text, self.__session, self)
-                )
-            except CourseUnavailable as exc:
-                if errors:
-                    raise CourseUnavailable(
-                        message=f"Course {li.a.text} in year {self.start}-{self.year} unavailable"
-                    ) from exc
-
-                print("Error with course", li.a.text)
-                continue
-
+        for course_info in courses_data:
+            if course_info.get("visible", False):
+                course_path = course_info["path"]
+                course_title = course_info["title"]
+                courses.append(Course(self.session, course_path, course_title, self))
         return courses
 
-    def get_course(self, name: str) -> Course:
+    def get_course(self, course_title: str) -> Course:
         """
-        get_course(self, name: str) -> Course
-        Gets a course by name.
+        Gets a course by its title.
         """
-        # Get the course
-        r = self.__session.get(self.url)
-        soup = BeautifulSoup(r.text, "lxml")
-        # Search by name
-        course = self.url + soup.find("a", text=name)["href"].replace(
-            f"course/{self.start}-{self.year}", ""
-        )
-        # Get the url and transform it into a course object
-        return Course(url=course, name=name, session=self.__session, parent=self)
+        all_courses = self.all_courses()
+        for course in all_courses:
+            if course.title == course_title:
+                return course
+        raise ValueError(f"Course '{course_title}' not found in year {self.year_path}.")
+
+    def get_course_by_tag(self, course_tag: str) -> Course:
+        """
+        Gets a course by its tag (course identifier).
+        Constructs the course URL using the year and course tag.
+        """
+        course_path = f"/{self.year_path}/{course_tag}"
+        course_url = f"{self.base_url}/course{course_path}"
+
+        response = self.session.get(course_url)
+        if response.status_code != 200:
+            raise ConnectionError(
+                f"Failed to retrieve course '{course_tag}' for year {self.year_path}."
+            )
+
+        soup = BeautifulSoup(response.text, "lxml")
+
+        title_elements = soup.find_all("a", class_="fill accent large")
+        title_element = title_elements[-1] if title_elements else None
+
+        if title_element:
+            course_title = title_element.get_text(strip=True)
+        else:
+            raise ValueError(
+                f"Could not retrieve course title for tag '{course_tag}' in year {self.year_path}."
+            )
+
+        return Course(self.session, course_path, course_title, self)
+
+    def __str__(self):
+        return f"Year({self.year_path})"