docs/api.md

@@ -17,8 +17,8 @@ On the first run, you will be prompted for your password. Then, on the next run(
 #### `login()`
 Logs in to Themis. Runs automatically when the class is initialized.
 
-#### `get_year(year_path)`
+#### `get_year(start, end)`
-Returns an instance of a [`Year`](#year) for the academic year specified by `year_path`.
+Returns an instance of a [`Year`](#year) (academic year) between `start` and `end`.
 
 ```python
 year = themis.get_year(2023, 2024)
@@ -30,7 +30,7 @@ Returns a list of `Year` instances corresponding to all years visible to the use
 ```python
 years = themis.all_years()
 ```
-
+<sub> I don't see why you would need this, but it's here. </sub>
 ----
 
 ## `Year`
@@ -41,20 +41,13 @@ year = themis.get_year(2023, 2024)
 ```
 
 ### Methods
-#### `get_course(course_title)`
+#### `get_course(name)`
-Returns an instance of a [`Course`](#course) with the title `course_title`.
+Returns an instance of a [`Course`](#course) with the name `name`.
 
 ```python
 pf = year.get_course("Programming Fundamentals (for CS)")
 ```
 
-#### `get_course_by_tag(course_tag)`
-Returns an instance of a [`Course`](#course) using the course identifier `course_tag`.
-
-```python
-ai_course = year.get_course_by_tag("adinc-ai")
-```
-
 #### `all_courses()`
 Returns a list of `Course` instances corresponding to all courses visible to the user in a given `Year`.
 
@@ -73,96 +66,160 @@ assignments = pf.get_groups()
 
 ### Methods
 #### `get_groups(full=False)`
-Returns a list of `ExerciseGroup` or `Group` instances corresponding to all items visible to the user in a given `Course`. The default argument is `full=False`, which will only return the top-level (name, link) of each item. If `full=True`, it will traverse the whole course.
+Returns a list of `ExerciseGroup` instances corresponding to all exercise groups visible to the user in a given `Course`. The default argument is `full=False`, which will only return the top-level (name, link) of each exercise and folder in the group. If `full=True`, it will traverse the whole course.
+
+You can traverse the course in both cases, although in different ways.
+
+When you have fully traversed the course, you can access everything via indices and the `exercises` and `folders` attributes of the `ExerciseGroup` instances:
 
 ```python
-ai_groups = ai_course.get_groups(full=True)
+ai_group = ai_course.get_groups(full=True)
-exercise = ai_groups[7].exercises[1]
+exercise = ai_group[7].exercises[1]  # Week 11 -> Suitcase packing
-exercise.submit(["solution.py"], silent=False)
+exercise.submit(["suitcase.py"], silent=False)
+```
+
+This is equivalent to the case in which we don't traverse the whole course using `get_group` like so:
+
+```python
+ai_group = ai_course.get_group("Week 11")
+exercise = ai_group.get_group("Suitcase packing")
+exercise.submit(["suitcase.py"], silent=False)
 ```
 
 #### `get_group(name, full=False)`
-Returns an instance of an `ExerciseGroup` or `Group` with the name `name`. The default argument is `full=False`, which will only return the (name, link) of the group. If `full=True`, it will traverse the whole group.
+Returns an instance of an `ExerciseGroup` with the name `name`. The default argument is `full=False`, which will only return the (name, link) of each exercise and folder in the group. If `full=True`, it will traverse the whole group.
 
 ```python
 week1 = pf.get_group("Week 1")
 ```
 
-#### `create_group(item_data)`
-Creates and returns a `Group` or `ExerciseGroup` instance based on `item_data`.
-
-```python
-group = course.create_group(item_data)
-```
-
-----
-
-## `Group`
-
-Represents an item in Themis, which can be either a folder (non-submittable) or an assignment (submittable).
-
-### Methods
-#### `get_items()`
-Returns all items (groups and assignments) under this group.
-
-```python
-items = week1.get_items()
-```
-
-#### `get_item_by_title(title)`
-Returns a single item by its title (case-insensitive).
-
-```python
-item = week1.get_item_by_title("Exercise 2")
-```
-
-#### `get_status(text=False)`
-Retrieves the status of the group. When `text=True`, returns the status as strings. Otherwise, returns submission objects or strings.
-
-```python
-status = group.get_status()
-leading_submission = status["leading"]
-```
-
-#### `download_files(path=".")`
-Downloads all files available for this group to a directory `path`. Defaults to the current directory.
-
-```python
-group.download_files()
-```
-
-#### `download_tcs(path=".")`
-Downloads all test cases for this group to a directory `path`. Defaults to the current directory.
-
-```python
-group.download_tcs()
-```
-
-#### `submit(files, judge=True, wait=True, silent=True)`
-Submits the files to the group. Default arguments are `judge=True`, `wait=True`, and `silent=True`.
-
-```python
-group.submit(["solution.py"], silent=False)
-```
-
 ----
 
 ## `ExerciseGroup`
-Represents a submittable exercise. Inherits from `Group`.
+Setting the `full` flag to `True` will traverse the whole group.
 
-### Additional Methods
+- Both folders and exercises are represented as `ExerciseGroup` instances.
-#### `submit(files)`
+- Folders will have the `am_exercise` attribute set to `False`.
-Submits files to the exercise. Raises an error if the item is not submittable.
+- Folders can have the `download_files` method called on them.
+- Exercises can have the `submit`, `download_files`, and `download_tcs` methods called on them.
+
+### Example of folder traversal
+Let's say we have a folder structure like this:
+```
+- Course Name
+  - Week 1
+    - Exercise 1
+    - Exercise 2
+      - Part 1
+      - Part 2
+  - Week 2
+    - Exercise 1
+    - Exercise 2
+```
+And we want to get to `Part 2` of `Week 1`'s `Exercise 2`. We would do this:
 
 ```python
-exercise.submit(["solution.py"])
+pf = year.get_course("Programming Fundamentals (for CS)")
+assignments = pf.get_groups()
+week1 = assignments[0]  # Week 1
+exercise2 = week1.folders[1]  # Exercise 2
+part2 = exercise2.exercises[1]  # Part 2
+
+# Or, if you don't want to traverse the whole course:
+week1 = pf.get_group("Week 1")
+exercise2 = week1.get_group("Exercise 2")
+part2 = exercise2.get_group("Part 2")
+```
+
+### Methods
+#### `download_files(path=".")`
+Downloads all files in the exercise group to a directory `path`. Defaults to the current directory.
+
+```python
+assignment.download_files()
+```
+
+#### `download_tcs(path=".")`
+Downloads all test cases in the exercise group to a directory `path`. Defaults to the current directory.
+
+```python
+assignment.download_tcs()
+```
+
+#### `get_group(name, full=False)`
+This is used when you want to traverse the course dynamically (not recurse through the whole thing). You can use it even if you've traversed the whole course.
+
+```python
+# Week 1 -> Exercise 2 -> Part 2
+week1 = pf.get_group("Week 1")
+exercise2 = week1.get_group("Exercise 2")
+part2 = exercise2.get_group("Part 2")
+
+# This is equivalent to (but faster than):
+week1 = pf.get_groups(full=True)[0]
+exercise2 = week1.folders[1]
+part2 = exercise2.exercises[1]
+```
+
+#### `submit(files, judge=True, wait=True, silent=True)`
+Submits the files to the exercise. The default arguments are `judge=True`, `wait=True`, and `silent=True`. Setting `judge=False` will not judge the submission immediately. Setting `wait=False` will not wait for the submission to finish. Turning off `silent` will print the submission status dynamically.
+
+```python
+suitcase = ai_course.get_group("Week 11").get_group("Suitcase packing")
+suitcase.submit(["suitcase.py"], silent=False)
+
+# Output:
+# Submitting to Suitcase packing
+# • suitcase.py
+# 1: ✅
+# 2: ✅
+# 3: ✅
+# ...
+```
+
+#### `get_status(text=False)`
+Retrieves the status of the exercise group. When `text` is set to `True`, it will return the status as a dictionary of strings. Otherwise, it will return a dictionary where keys map to either strings or `Submission` objects. Common keys include `'leading'`, `'best'`, `'latest'`, etc.
+
+```python
+pf = year.get_course("Programming Fundamentals (for CS)")
+exercise = pf.get_group("Lab Session 2").get_group("Recurrence")
+
+# Get status
+status = exercise.get_status()
+print(status)
+
+# Output:
+{
+  'assignment': 'Recurrence',
+  'group': 'Y.N. Here',
+  'status': 'passed: Passed all test cases',
+  'grade': '2.00',
+  'total': '2',
+  'output limit': '1',
+  'passed': '1',
+  'leading': <temmies.submission.Submission object at 0x...>,
+  'best': <temmies.submission.Submission object at 0x...>,
+  'latest': <temmies.submission.Submission object at 0x...>,
+  'first_pass': <temmies.submission.Submission object at 0x...>,
+  'last_pass': <temmies.submission.Submission object at 0x...>,
+  'visible': 'Yes'
+}
+```
+
+To access submission details:
+
+```python
+leading_submission = status["leading"]
+print(leading_submission.get_files())
 ```
 
 ----
 
 ## `Submission`
-
+### Usage
-Represents a submission for a specific exercise.
+```python
+submission = pf.get_group("Week 1").get_group("Exercise 1").get_group("Part 1").get_status()["leading"]
+```
 
 ### Methods
 #### `get_test_cases()`
@@ -170,13 +227,33 @@ Returns a dictionary of test cases and their statuses.
 
 ```python
 test_cases = submission.get_test_cases()
+print(test_cases)
+
+# Output:
+{'1': 'passed', '2': 'passed', '3': 'passed', '4': 'passed', '5': 'passed', '6': 'passed', '7': 'passed', '8': 'passed', '9': 'passed', '10': 'passed'}
 ```
 
 #### `get_info()`
-Returns detailed information about the submission.
+Returns a dictionary of information about the submission.
 
 ```python
 info = submission.get_info()
+print(info)
+
+# Output:
+{
+  'assignment': 'Part 1',
+  'group': 'Y.N. Here',
+  'uploaded_by': 'Y.N. Here s1234567',
+  'created_on': 'Wed Sep 13 2023 12:51:37 GMT+0200',
+  'submitted_on': 'Wed Sep 13 2023 12:51:37 GMT+0200',
+  'status': 'passed: Passed all test cases',
+  'files': [
+    ('recurrence.c', '/file/.../recurrence.c'),
+    ('compile.log', '/file/.../compile.log')
+  ],
+  'language': 'c'
+}
 ```
 
 #### `get_files()`
@@ -184,4 +261,13 @@ Returns a list of uploaded files in the format `(name, URL)`.
 
 ```python
 files = submission.get_files()
+print(files)
+
+# Output:
+[
+  ('recurrence.c', '/file/.../recurrence.c'),
+  ('compile.log', '/file/.../compile.log')
+]
 ```
+
+----

docs/changelog.md

@@ -11,11 +11,4 @@
 #### **Codebase**
 - Prepended `get_` to all methods in `Submission`
 - Created base `Group` from which `Course` and `ExerciseGroup` inherit.
-- Using system keyring to store passwords (Issue #11)
+- Using system keyring to store passwords (Issue #11)
-
-### **Version 1.2.0**
-
-#### **Codebase**
-- Moved all methods related to downloading files (including test cases) to `Group`.
-- Created `get_test_cases` and `get_files` methods in `Group`.
-- We are now using the [API](https://themis.housing.rug.nl/api/navigation/2023-2024) (which mysteriously appeared) to get the year/course structure.

docs/index.md

@@ -24,7 +24,7 @@ from temmies.themis import Themis
 themis = Themis("s-number") # You will be prompted for your password
 
 # Get a year
-year = themis.get_year("2023-2024")
+year = themis.get_year(2023, 2024)
 
 # Get a course
 course = year.get_course("Programming Fundamentals (for CS)")

temmies/__init__.py

@@ -1,5 +1,3 @@
 from .themis import Themis
-import urllib3
 
 __all__ = ["Themis"]
-urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

temmies/course.py

@@ -1,35 +1,28 @@
 from .group import Group
-
+from .exercise_group import ExerciseGroup
+from requests import Session
+from .exceptions.course_unavailable import CourseUnavailable
 
 class Course(Group):
     """
-    Represents a course.
+    Represents a course in a given academic year.
     """
 
-    def __init__(self, session, course_path: str, title: str, parent):
+    def __init__(self, url: str, name: str, session, parent):
-        super().__init__(session, course_path, title, parent)
+        super().__init__(url, name, session, parent=parent, full=False)
-        self.course_path = course_path  # e.g., '/2023-2024/adinc-ai'
+        self.__course_available(self._request)
 
     def __str__(self):
-        return f"Course({self.title})"
+        return f"Course {self.name} in year {self._parent.year}"
 
-    def create_group(self, item_data):
+    def __course_available(self, response):
-        """
+        if "Something went wrong" in response.text:
-        Create a subgroup (Group or ExerciseGroup) based on item data.
+            raise CourseUnavailable(
-        """
+                message="'Something went wrong'. Course most likely not found."
-        if item_data.get("submitable", False):
-            return ExerciseGroup(
-                self.session,
-                item_data["path"],
-                item_data["title"],
-                self,
-                item_data.get("submitable", False),
-            )
-        else:
-            return Group(
-                self.session,
-                item_data["path"],
-                item_data["title"],
-                self,
-                item_data.get("submitable", False),
             )
+
+    def create_group(self, url: str, name: str, session: Session, parent, full: bool, classes=None):
+        """
+        Create an instance of ExerciseGroup for subgroups within a Course.
+        """
+        return ExerciseGroup(url, name, session, parent, full, classes)

temmies/exercise_group.py

@@ -1,51 +1,182 @@
 from .group import Group
+from .exceptions.illegal_action import IllegalAction
 from .submission import Submission
+from json import loads
+from time import sleep
+from typing import Optional
 from bs4 import BeautifulSoup
 
 class ExerciseGroup(Group):
     """
-    Represents a submittable exercise.
+    Represents a group of exercises or a single exercise.
     """
 
-    def __init__(self, session, path: str, title: str, parent, submitable: bool = True):
+    def __init__(self, url: str, name: str, session, parent=None, full: bool = False, classes=None):
-        super().__init__(session, path, title, parent, submitable=submitable)
+        super().__init__(url, name, session, parent=parent, full=full, classes=classes)
-        self.submit_url = f"{self.base_url}/api/submit{self.path}"
+        self.am_exercise = "ass-submitable" in self.classes
-        self.__find_name()
+
-    
+    def create_group(self, url: str, name: str, session, parent, full: bool, classes=None):
-    def __find_name(self):
         """
-        Find the name of the exercise group.
+        Create an instance of ExerciseGroup for subgroups.
         """
-        if self.title == "":
+        return ExerciseGroup(url, name, session, parent, full, classes)
-            # Find using beautiful soup (it is the last a with class 'fill accent large')
+
-            response = self.session.get(self.base_url + self.path)
+    @property
-            soup = BeautifulSoup(response.text, "lxml")
+    def test_cases(self) -> list[str]:
-            title_elements = soup.find_all("a", class_="fill accent large")
+        """
-            if title_elements:
+        Get all test cases for this exercise.
-                self.title = title_elements[-1].get_text(strip=True)
+        """
-            else:
+        if not self.am_exercise:
-                self.title = self.path.split("/")[-1]
+            return []
-                
+
-    def submit(self, files: list[str]) -> Submission:
+        sections = self._raw.find_all("div", class_="subsec round shade")
+        tcs = []
+        for div in sections:
+            res = div.find("h4", class_="info")
+            if res and "Test cases" in res.text:
+                for case in div.find_all("div", class_="cfg-line"):
+                    if link := case.find("a"):
+                        tcs.append(link)
+        return tcs
+
+    def download_tcs(self, path=".") -> list[str]:
+        """
+        Download all test cases for this exercise.
+        """
+        if not self.am_exercise:
+            raise IllegalAction("You are downloading test cases from a folder.")
+
+        for tc in self.test_cases:
+            url = f"https://themis.housing.rug.nl{tc['href']}"
+            print(f"Downloading {tc.text}")
+            with open(f"{path}/{tc.text}", "wb") as f:
+                f.write(self._session.get(url).content)
+        return self.test_cases
+
+    @property
+    def files(self) -> list[str]:
+        """
+        Get all downloadable files for this exercise or group.
+        """
+        details = self._raw.find("div", id=lambda x: x and x.startswith("details"))
+        if not details:
+            return []
+
+        cfg_lines = details.find_all("div", class_="cfg-line")
+        link_list = []
+
+        for line in cfg_lines:
+            key = line.find("span", class_="cfg-key")
+            if key and "Downloads" in key.text.strip():
+                links = line.find_all("span", class_="cfg-val")
+                for link in links:
+                    a_tags = link.find_all("a")
+                    for a in a_tags:
+                        link_list.append(a)
+        return link_list
+
+    def download_files(self, path=".") -> list[str]:
+        """
+        Download all files available for this exercise or group.
+        """
+        for file in self.files:
+            print(f"Downloading file {file.text}")
+            url = f"https://themis.housing.rug.nl{file['href']}"
+            with open(f"{path}/{file.text}", "wb") as f:
+                f.write(self._session.get(url).content)
+        return self.files
+
+    def submit(self, files: list[str], judge: bool = True, wait: bool = True, silent: bool = True) -> Optional[dict]:
         """
         Submit files to this exercise.
+        Returns a dictionary of test case results or None if wait is False.
         """
-        if not self.submitable:
+        if not self.am_exercise:
-            raise ValueError(f"Cannot submit to non-submittable item '{self.title}'.")
+            raise IllegalAction("You cannot submit to this assignment.")
 
-        # Prepare the files and data for submission
+        form = self._raw.find("form")
-        files_payload = {}
+        if not form:
-        for idx, file_path in enumerate(files):
+            raise IllegalAction("Submission form not found.")
-            file_key = f"file{idx}"
-            with open(file_path, "rb") as f:
-                files_payload[file_key] = (file_path, f.read())
 
-        response = self.session.post(self.submit_url, files=files_payload)
+        url = "https://themis.housing.rug.nl" + form["action"]
-        if response.status_code != 200:
+        file_types = loads(form["data-suffixes"])
-            raise ConnectionError(f"Failed to submit to '{self.title}'.")
 
-        submission_data = response.json()
+        if isinstance(files, str):
-        return Submission(self.session, submission_data)
+            files = [files]
 
-    def __str__(self):
+        packaged_files = []
-        return f"ExerciseGroup({self.title})"
+        data = {}
+        found_type = ""
+
+        for file in files:
+            for suffix, lang in file_types.items():
+                if file.endswith(suffix):
+                    found_type = lang
+                    break
+            if not found_type:
+                print("WARNING: File type not recognized")
+
+            with open(file, "rb") as f:
+                packaged_files.append((found_type, (file, f.read())))
+
+        data = {
+            "judgenow": "true" if judge else "false",
+            "judgeLanguage": found_type if found_type else "none"
+        }
+
+        if not silent:
+            print(f"Submitting to {self.name}")
+            for file in files:
+                print(f"• {file}")
+
+        resp = self._session.post(url, files=packaged_files, data=data)
+
+        if not wait or not judge:
+            return resp.url if "@submissions" in resp.url else None
+
+        return self.__wait_for_result(resp.url, not silent, [])
+
+    def __wait_for_result(self, url: str, verbose: bool, __printed: list) -> dict:
+        """
+        Wait for the submission result and return the test case results.
+        """
+        r = self._session.get(url)
+        soup = BeautifulSoup(r.text, "lxml")
+        return self.__parse_table(soup, url, verbose, __printed)
+
+    def __parse_table(self, soup: BeautifulSoup, url: str, verbose: bool, __printed: list) -> dict:
+        """
+        Parse the results table from the submission result page.
+        """
+        cases = soup.find_all("tr", class_="sub-casetop")
+        fail_pass = {}
+        for case in cases:
+            name = case.find("td", class_="sub-casename").text
+            status = case.find("td", class_="status-icon")
+
+            if "pending" in status.get("class"):
+                sleep(1)
+                return self.__wait_for_result(url, verbose, __printed)
+
+            statuses = {
+                "Passed": ("✅", True),
+                "Wrong output": ("❌", False),
+                "No status": ("🐛", None),
+                "error": ("🐛", None),
+            }
+
+            found = False
+            for k, v in statuses.items():
+                if k in status.text:
+                    found = True
+                    if verbose and int(name) not in __printed:
+                        print(f"{name}: {v[0]}")
+                    fail_pass[int(name)] = v[1]
+                    break
+            if not found:
+                fail_pass[int(name)] = None
+                if verbose and int(name) not in __printed:
+                    print(f"{name}: Unrecognized status: {status.text}")
+
+            __printed.append(int(name))
+        return fail_pass

temmies/group.py

@@ -1,88 +1,98 @@
+# temmies/group.py
+
 from bs4 import BeautifulSoup
 from requests import Session
-import os
 from typing import Optional, Union, Dict
 from .exceptions.illegal_action import IllegalAction
 from .submission import Submission
 
+
 class Group:
     """
-    Represents an item in Themis, which can be either a folder (non-submittable) or an assignment (submittable).
+    Base class for Course and ExerciseGroup.
     """
 
-    def __init__(self, session, path: str, title: str, parent=None, submitable: bool = False):
+    def __init__(self, url: str, name: str, session: Session, parent=None, full: bool = False, classes=None):
-        self.session = session
+        self.url = url
-        self.path = path  # e.g., '/2023-2024/adinc-ai/labs'
+        self.name = name
-        self.title = title
+        self._session = session
-        self.parent = parent
+        self._parent = parent
-        self.submitable = submitable
+        self._full = full
-        self.base_url = "https://themis.housing.rug.nl"
+        self._request = self._session.get(self.url)
-        self.api_url = f"{self.base_url}/api/navigation{self.path}"
+        self._raw = BeautifulSoup(self._request.text, "lxml")
-        self.classes = []
+        self.classes = classes or []
 
-        # Adjust URL construction to include '/course' when accessing HTML pages
+    def __str__(self):
-        if not self.path.startswith('/course/'):
+        return f"Group {self.name}"
-            group_url = f"{self.base_url}/course{self.path}"
-        else:
-            group_url = f"{self.base_url}{self.path}"
 
-        # Fetch the page and parse it
+    def get_groups(self, full: bool = False):
-        response = self.session.get(group_url)
-        if response.status_code != 200:
-            raise ConnectionError(f"Failed to retrieve page for '{self.title}'. Tried {group_url}")
-        self._raw = BeautifulSoup(response.text, "lxml")
-
-
-    def get_items(self) -> list:
         """
-        Get all items (groups and assignments) under this group.
+        Get all groups (exercises and folders) within this group.
         """
         section = self._raw.find("div", class_="ass-children")
         if not section:
             return []
 
         entries = section.find_all("a", href=True)
-        items = []
+        groups = []
         for x in entries:
             href = x['href']
             name = x.text.strip()
             classes = x.get('class', [])
-            submitable = "ass-submitable" in classes
+            group = self.create_group(
-            item = Group(
+                url=f"https://themis.housing.rug.nl{href}",
-                session=self.session,
+                name=name,
-                path=href,
+                session=self._session,
-                title=name,
                 parent=self,
-                submitable=submitable
+                full=full,
+                classes=classes
             )
-            items.append(item)
+            groups.append(group)
-        return items
+        return groups
 
-    def get_item_by_title(self, title: str):
+    def get_group(self, name: str, full: bool = False):
         """
-        Get a single item by its title, case-insensitive.
+        Get a single group by name.
         """
-        items = self.get_items()
+        group_link = self._raw.find("a", text=name)
-        for item in items:
+        if not group_link:
-            if (item.title.lower() == title.lower()) or (item.path.split("/")[-1] == title):
+            raise IllegalAction(f"No such group found: {name}")
-                return item
+        href = group_link['href']
-        raise ValueError(f"Item '{title}' not found under {self.title}.")
+        classes = group_link.get('class', [])
+        return self.create_group(
+            url=f"https://themis.housing.rug.nl{href}",
+            name=name,
+            session=self._session,
+            parent=self,
+            full=full,
+            classes=classes
+        )
 
+    def create_group(self, url: str, name: str, session: Session, parent, full: bool, classes=None):
+        """
+        Factory method to create a group. Subclasses must implement this.
+        """
+        raise NotImplementedError("Subclasses must implement create_group")
 
-    def get_status(self, text: bool = False) -> Union[Dict[str, Union[str, 'Submission']], None]:
+    def get_status(self, text: bool = False) -> Union[Dict[str, Union[str, Submission]], None]:
         """
         Get the status of the current group, if available.
+
+        Args:
+            text (bool): If True, returns text representation of the status.
+                        Otherwise, creates `Submission` objects for applicable fields.
+
+        Returns:
+            dict[str, Union[str, Submission]] | None: The status data for the group, 
+                                                    with `Submission` objects for links.
         """
         status_link = self._raw.find("a", text="Status")
         if not status_link:
-            raise ValueError("Status information is not available for this group.")
+            raise IllegalAction("Status information is not available for this group.")
 
-        status_url = f"{self.base_url}{status_link['href']}"
+        status_url = f"https://themis.housing.rug.nl{status_link['href']}"
-        response = self.session.get(status_url)
+        r = self._session.get(status_url)
-        if response.status_code != 200:
+        soup = BeautifulSoup(r.text, "lxml")
-            raise ConnectionError(f"Failed to retrieve status page for '{self.title}'.")
-
-        soup = BeautifulSoup(response.text, "lxml")
         section = soup.find("div", class_="cfg-container")
 
         if not section:
@@ -90,9 +100,17 @@ class Group:
 
         return self.__parse_status_section(section, text)
 
-    def __parse_status_section(self, section: BeautifulSoup, text: bool) -> Dict[str, Union[str, 'Submission']]:
+    def __parse_status_section(self, section: BeautifulSoup, text: bool) -> Dict[str, Union[str, Submission]]:
         """
         Parse the status section of the group and clean up keys.
+
+        Args:
+            section (BeautifulSoup): The HTML section containing the status information.
+            text (bool): Whether to return text representation.
+
+        Returns:
+            dict[str, Union[str, Submission]]: Parsed and cleaned status information,
+                                            with `Submission` objects for links.
         """
         key_mapping = {
             "leading the submission that counts towards the grade": "leading",
@@ -117,202 +135,9 @@ class Group:
             # Process value
             link = value_element.find("a", href=True)
             if link and not text:
-                href = link["href"]
+                submission_url = link["href"]
-                # Construct full URL
+                parsed[key] = Submission(submission_url, self._session)
-                if href.startswith("/"):
-                    submission_url = href
-                elif href.startswith("http"):
-                    submission_url = href.replace("https://themis.housing.rug.nl", "")
-                else:
-                    print(f"Invalid href '{href}' found in status page.")
-                    continue  # Skip this entry if href is invalid
-
-                # Instantiate Submission with submission_url and session
-                submission = Submission(submission_url, self.session)
-                parsed[key] = submission
             else:
                 parsed[key] = value_element.get_text(separator=" ").strip()
 
         return parsed
-
-
-    def get_test_cases(self) -> list[Dict[str, str]]:
-        """
-        Get all test cases for this assignment.
-        """
-        if not self.submitable:
-            raise ValueError(f"No test cases for non-submittable item '{self.title}'.")
-
-        sections = self._raw.find_all("div", class_="subsec round shade")
-        tcs = []
-        for div in sections:
-            res = div.find("h4", class_="info")
-            if res and "Test cases" in res.text:
-                for case in div.find_all("div", class_="cfg-line"):
-                    link = case.find("a")
-                    if link:
-                        tcs.append({
-                            'title': link.text.strip(),
-                            'path': link['href']
-                        })
-        return tcs
-
-    def download_tcs(self, path=".") -> list[str]:
-        """
-        Download all test cases for this assignment.
-        """
-        test_cases = self.get_test_cases()
-        downloaded = []
-        for tc in test_cases:
-            url = f"{self.base_url}{tc['path']}"
-            print(f"Downloading {tc['title']}")
-            response = self.session.get(url)
-            if response.status_code == 200:
-                tc_filename = os.path.join(path, tc['title'])
-                with open(tc_filename, 'wb') as f:
-                    f.write(response.content)
-                downloaded.append(tc_filename)
-            else:
-                print(f"Failed to download test case '{tc['title']}'")
-        return downloaded
-
-    def get_files(self) -> list[Dict[str, str]]:
-        """
-        Get all downloadable files for this assignment.
-        """
-        details = self._raw.find("div", id=lambda x: x and x.startswith("details"))
-        if not details:
-            return []
-
-        cfg_lines = details.find_all("div", class_="cfg-line")
-        files = []
-
-        for line in cfg_lines:
-            key = line.find("span", class_="cfg-key")
-            if key and "Downloads" in key.text.strip():
-                vals = line.find_all("span", class_="cfg-val")
-                for val in vals:
-                    links = val.find_all("a")
-                    for link in links:
-                        files.append({
-                            'title': link.text.strip(),
-                            'path': link['href']
-                        })
-        return files
-
-    def download_files(self, path=".") -> list[str]:
-        """
-        Download all files available for this assignment.
-        """
-        files = self.get_files()
-        downloaded = []
-        for file in files:
-            print(f"Downloading file '{file['title']}'")
-            url = f"{self.base_url}{file['path']}"
-            response = self.session.get(url)
-            if response.status_code == 200:
-                file_filename = os.path.join(path, file['title'])
-                with open(file_filename, 'wb') as f:
-                    f.write(response.content)
-                downloaded.append(file_filename)
-            else:
-                print(f"Failed to download file '{file['title']}'")
-        return downloaded
-
-    def submit(self, files: list[str], judge: bool = True, wait: bool = True, silent: bool = True) -> Optional[dict]:
-        """
-        Submit files to this assignment.
-        Returns a dictionary of test case results or None if wait is False.
-        """
-        if not self.submitable:
-            raise ValueError(f"Cannot submit to non-submittable item '{self.title}'.")
-
-        form = self._raw.find("form")
-        if not form:
-            raise ValueError("Submission form not found.")
-
-        url = f"{self.base_url}{form['action']}"
-        file_types = loads(form.get("data-suffixes", "{}"))
-
-        if isinstance(files, str):
-            files = [files]
-
-        packaged_files = []
-        data = {}
-        found_type = ""
-
-        for file in files:
-            for suffix, lang in file_types.items():
-                if file.endswith(suffix):
-                    found_type = lang
-                    break
-            if not found_type:
-                print("WARNING: File type not recognized")
-
-            with open(file, "rb") as f:
-                packaged_files.append((found_type, (file, f.read())))
-
-        data = {
-            "judgenow": "true" if judge else "false",
-            "judgeLanguage": found_type if found_type else "none"
-        }
-
-        if not silent:
-            print(f"Submitting to {self.title}")
-            for file in files:
-                print(f"• {file}")
-
-        resp = self.session.post(url, files=packaged_files, data=data)
-
-        if not wait or not judge:
-            return resp.url if "@submissions" in resp.url else None
-
-        return self.__wait_for_result(resp.url, not silent, [])
-
-    def __wait_for_result(self, url: str, verbose: bool, __printed: list) -> dict:
-        """
-        Wait for the submission result and return the test case results.
-        """
-        r = self.session.get(url)
-        soup = BeautifulSoup(r.text, "lxml")
-        return self.__parse_table(soup, url, verbose, __printed)
-
-    def __parse_table(self, soup: BeautifulSoup, url: str, verbose: bool, __printed: list) -> dict:
-        """
-        Parse the results table from the submission result page.
-        """
-        cases = soup.find_all("tr", class_="sub-casetop")
-        fail_pass = {}
-        for case in cases:
-            name = case.find("td", class_="sub-casename").text
-            status = case.find("td", class_="status-icon")
-
-            if "pending" in status.get("class"):
-                sleep(1)
-                return self.__wait_for_result(url, verbose, __printed)
-
-            statuses = {
-                "Passed": ("✅", True),
-                "Wrong output": ("❌", False),
-                "No status": ("🐛", None),
-                "error": ("🐛", None),
-            }
-
-            found = False
-            for k, v in statuses.items():
-                if k in status.text:
-                    found = True
-                    if verbose and int(name) not in __printed:
-                        print(f"{name}: {v[0]}")
-                    fail_pass[int(name)] = v[1]
-                    break
-            if not found:
-                fail_pass[int(name)] = None
-                if verbose and int(name) not in __printed:
-                    print(f"{name}: Unrecognized status: {status.text}")
-
-            __printed.append(int(name))
-        return fail_pass
-
-    def __str__(self):
-        return f"Group({self.title}, submitable={self.submitable})"

temmies/submission.py

@@ -1,7 +1,5 @@
-# submission.py
-
 """
-File to define the Submission class
+File to define the submission class
 """
 
 from bs4 import BeautifulSoup
@@ -9,11 +7,11 @@ from bs4 import BeautifulSoup
 class Submission:
     """
     Submission class
-
+    
     Methods:
-    get_test_cases: Get a dict of test cases status
+    test_cases: Get a dict of test cases status
-    get_info: Submission information (in details)
+    info: Submission information (in details)
-    get_files: Get a list of uploaded files (as names)
+    files: Get a list of uploaded files(as names)
     """
     def __init__(self, url: str, session):
         self.url = "https://themis.housing.rug.nl" + url
@@ -26,17 +24,23 @@ class Submission:
         """Clean text"""
         clean = text.replace("\t", "").replace("\n", "")
         if value:
-            return clean.strip()
+            return clean
-        return clean.replace(" ", "_").replace(":", "").lower().strip()
+        return clean.replace(" ", "_").replace(":", "").lower()
 
     def get_test_cases(self) -> dict[str, str]:
         """Get a dict of test cases status"""
+        # In the submission page, the test cases are in a div with class "sub-cases subsec round shade"
+        # print(self.__raw.prettify())
         cases = self.__raw.find("div", class_=lambda x: x and "sub-cases" in x.split())
         if not cases:
             return {}
 
+        # The test cases are in a table in a div with class "cfg-container"
         cases = cases.find("div", class_="cfg-container")
         cases = cases.find("table")
+        # For each test case, there is a tr with class sub-casetop, which contains 2 tds:
+        # * a td with class "sub-case name" which is a name
+        # * a td with a variable class, which is the status text
 
         results = {}
         for entry in cases.find_all("tr", class_="sub-casetop"):
@@ -44,51 +48,37 @@ class Submission:
             status = entry.find(
                 "td", class_=lambda x: x and "status-icon" in x.split()
             ).text
-            results[name.strip()] = self.__clean(status)
+            results[name] = self.__clean(status)
 
         return results
 
     def get_info(self) -> dict[str, str] | None:
         """Submission information (in details)"""
+        # in div with class subsec round shade where there is an h4 with class info
+        # The info is in a div with class "cfg-container"
         if self.__info:
             return self.__info
 
         for div in self.__raw.find_all("div", class_="subsec round shade"):
-            h4 = div.find("h4", class_=lambda x: x and "info" in x.split())
+            if h4 := div.find("h4", class_=lambda x: x and "info" in x.split()):
-            if h4 and "Details" in h4.text:
+                if "Details" in h4.text:
-                info = div.find("div", class_="cfg-container")
+                    # The information is in divs with class "cfg-line"
-                info_lines = info.find_all("div", class_="cfg-line")
+                    # With key in span with class "cfg-key" and value in span with class "cfg-value"
-                self.__info = {
+                    info = div.find("div", class_="cfg-container")
-                    self.__clean(
+                    info = info.find_all("div", class_="cfg-line")
-                        key := line.find("span", class_="cfg-key").text
+                    return {
-                    ): (
+                        self.__clean(
-                        self.__clean(line.find("span", class_="cfg-val").text, value=True)
+                            key := line.find("span", class_="cfg-key").text
-                        if "Files" not in key
+                        ): 
-                        else [
+                            self.__clean(line.find("span", class_="cfg-val").text, value=True) if "Files" not in key else 
-                            (self.__clean(a.text), a["href"])
+                            ([(self.__clean(x.text), x["href"]) for x in line.find("span", class_="cfg-val").find_all("a")])
-                            for a in line.find("span", class_="cfg-val").find_all("a")
+                        for line in info
-                        ]
+                    }
-                    )
-                    for line in info_lines
-                }
-                return self.__info
         return None
 
     def get_files(self) -> list[str] | None:
         """Get a list of uploaded files in the format [(name, url)]"""
         if not self.__info:
             self.__info = self.get_info()
+        
         return self.__info.get("files", None)
-
-    # Deprecated methods
-    def info(self):
-        print("This method is deprecated and will be deleted soon. Use get_info instead.")
-        return self.get_info()
-
-    def test_cases(self):
-        print("This method is deprecated and will be deleted in soon. Use get_test_cases instead.")
-        return self.get_test_cases()
-
-    def files(self):
-        print("This method is deprecated and will be deleted in soon. Use get_files instead.")
-        return self.get_files()

temmies/themis.py

@@ -1,7 +1,9 @@
 """
-Main class for the Themis API using the new JSON endpoints.
+Main class for the Themis API
+
 """
 
+import urllib3
 import keyring
 import getpass
 from requests import Session
@@ -9,49 +11,41 @@ from bs4 import BeautifulSoup
 from .year import Year
 from .exceptions.illegal_action import IllegalAction
 
+
+# Disable warnings
+urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
+
+
 class Themis:
     """
-    Main class for interacting with Themis.
+    login: Login to Themis
-    - login: Login to Themis
+    get_year: Get a year object
-    - get_year: Get a year object
+    all_years: Get all years
-    - all_years: Get all years
     """
 
     def __init__(self, user: str):
-        """
-        Initialize Themis object, logging in with the given user.
-
-        Args:
-            user (str): Username to login with.
-
-        Attributes:
-            user (str): Username.
-            password (str): Password, retrieved from keyring.
-            base_url (str): Base URL of the Themis website.
-            session (requests.Session): Authenticated session.
-        """
         self.user = user
         self.password = self.__get_password()
-        self.base_url = "https://themis.housing.rug.nl"
+        self.session = self.login(user, self.password)
-        self.session = self.login(self.user, self.password)
+
     def __get_password(self) -> str:
         """
         Retrieve the password from the keyring, prompting the user if not found.
         """
-        password = keyring.get_password(f"{self.user}-temmies", self.user)
+        password = keyring.get_password(f'{self.user}-temmies', self.user)
         if not password:
             print(f"Password for user '{self.user}' not found in keyring.")
             password = getpass.getpass(prompt=f"Enter password for {self.user}: ")
-            keyring.set_password(f"{self.user}-temmies", self.user, password)
+            keyring.set_password(f'{self.user}-temmies', self.user, password)
             print("Password saved securely in keyring.")
         return password
 
     def login(self, user: str, passwd: str) -> Session:
         """
-        Login to Themis using the original method, parsing CSRF token from the login page.
+        login(self, user: str, passwd: str) -> Session
+        Login to Themis
+        Set user to your student number and passwd to your password
         """
-        session = Session()
-        login_url = f"{self.base_url}/log/in"
 
         user_agent = (
             "Mozilla/5.0 (X11; Linux x86_64) "
@@ -63,54 +57,52 @@ class Themis:
 
         data = {"user": user, "password": passwd, "null": None}
 
-        # Get login page to retrieve CSRF token
+        with Session() as s:
-        response = session.get(login_url, headers=headers, verify=False)
+            url = "https://themis.housing.rug.nl/log/in"
-        if response.status_code != 200:
+            r = s.get(url, headers=headers, verify=False)
-            raise ConnectionError("Failed to connect to Themis login page.")
+            soup = BeautifulSoup(r.text, "lxml")
 
-        # Parse CSRF token from login page
+            # get the csrf token and add it to payload
-        soup = BeautifulSoup(response.text, "lxml")
+            csrf_token = soup.find("input", attrs={"name": "_csrf"})["value"]
-        csrf_input = soup.find("input", attrs={"name": "_csrf"})
+            data["_csrf"] = csrf_token
-        if not csrf_input or not csrf_input.get("value"):
+            data["sudo"] = user.lower()
-            raise ValueError("Unable to retrieve CSRF token.")
-        csrf_token = csrf_input["value"]
-        data["_csrf"] = csrf_token
-        data["sudo"] = user.lower()
 
-        # Attempt login
+            # Login
-        response = session.post(login_url, data=data, headers=headers)
+            r = s.post(url, data=data, headers=headers)
-        if "Invalid credentials" in response.text:
-            # Prompt for password again
-            print("Invalid credentials. Please try again.")
-            passwd = getpass.getpass(prompt="Enter password: ")
-            keyring.set_password(f'{self.user}-temmies', self.user, passwd)
-            return self.login(user, passwd)
-        elif "Welcome, logged in as" not in response.text:
-            raise ValueError("Login failed for an unknown reason.")
 
-        return session
+            # check if login was successful
+            log_out = "Welcome, logged in as" in r.text
+            if "Invalid credentials" in r.text:
+                # Prompt for password again
+                print("Invalid credentials. Please try again.")
+                passwd = getpass.getpass(prompt="Enter password: ")
+                keyring.set_password(f'{self.user}-temmies', self.user, passwd)
+                return self.login(user, passwd)
+                
+        return s
 
-    def get_year(self, start_year: int = None, end_year: int = None) -> Year:
+    def get_year(self, start: int, end: int) -> Year:
         """
-        Gets a Year object using the year path (e.g., 2023, 2024).
+        get_year(self, start: int, end: int) -> Year
+        Gets a year object
+        Set start to the start year and end to the end year (e.g. 2023-2024)
         """
-        year_path = f"{start_year}-{end_year}"
+        return Year(self.session, start, end)
-        
-        return Year(self.session, year_path)
 
-    def all_years(self) -> list:
+    def all_years(self) -> list[Year]:
         """
-        Gets all visible years as Year objects.
+        get_years(self, start: int, end: int) -> list[Year]
+        Gets all visible years
         """
-        navigation_url = f"{self.base_url}/api/navigation/"
+        # All of them are in a big ul at the beginning of the page
-        response = self.session.get(navigation_url)
+        r = self.session.get(self.url)
-        if response.status_code != 200:
+        soup = BeautifulSoup(r.text, "lxml")
-            raise ConnectionError("Failed to retrieve years from Themis API.")
+        ul = soup.find("ul", class_="round")
-
+        lis = ul.find_all("li", class_="large")
-        years_data = response.json()
         years = []
-        for year_info in years_data:
+        for li in lis:
-            if year_info.get("visible", False):
+            # format: 2019-2020
-                year_path = year_info["path"].strip("/")
+            year = li.a.text.split("-")
-                years.append(Year(self.session, year_path))
+            years.append(Year(self.session, int(year[0]), int(year[1])))
-        return years
+
+        return years  # Return a list of year objects

temmies/year.py

@@ -1,70 +1,52 @@
-from .course import Course
 from bs4 import BeautifulSoup
+from .course import Course
+from .exceptions.course_unavailable import CourseUnavailable
+
 class Year:
     """
     Represents an academic year.
     """
-    def __init__(self, session, year_path: str):
-        self.session = session
-        self.year_path = year_path  # e.g., '2023-2024'
-        self.base_url = "https://themis.housing.rug.nl"
-        self.api_url = f"{self.base_url}/api/navigation/{self.year_path}"
 
-    def all_courses(self) -> list:
+    def __init__(self, session, start_year: int, end_year: int):
-        """
+        self.start = start_year
-        Gets all visible courses in this year.
+        self.year = end_year
-        """
+        self.url = f"https://themis.housing.rug.nl/course/{self.start}-{self.year}"
-        response = self.session.get(self.api_url)
+        self._session = session
-        if response.status_code != 200:
-            raise ConnectionError(f"Failed to retrieve courses for {self.year_path}.")
 
-        courses_data = response.json()
+    def all_courses(self, errors: bool = True) -> list[Course]:
+        """
+        Gets all visible courses in a year.
+        """
+        r = self._session.get(self.url)
+        soup = BeautifulSoup(r.text, "lxml")
+        lis = soup.find_all("li", class_="large")
         courses = []
-        for course_info in courses_data:
+        for li in lis:
-            if course_info.get("visible", False):
+            try:
-                course_path = course_info["path"]
+                suffix = li.a["href"].replace(f"course/{self.start}-{self.year}", "")
-                course_title = course_info["title"]
+                course_url = self.url + suffix
-                courses.append(Course(self.session, course_path, course_title, self))
+                course_name = li.a.text.strip()
+                courses.append(
+                    Course(course_url, course_name, self._session, self)
+                )
+            except CourseUnavailable as exc:
+                if errors:
+                    raise CourseUnavailable(
+                        message=f"Course {li.a.text} in year {self.start}-{self.year} unavailable"
+                    ) from exc
+                print("Error with course", li.a.text)
+                continue
         return courses
 
-    def get_course(self, course_title: str) -> Course:
+    def get_course(self, name: str) -> Course:
         """
-        Gets a course by its title.
+        Gets a course by name.
         """
-        all_courses = self.all_courses()
+        r = self._session.get(self.url)
-        for course in all_courses:
+        soup = BeautifulSoup(r.text, "lxml")
-            if course.title == course_title:
+        course_link = soup.find("a", text=name)
-                return course
+        if not course_link:
-        raise ValueError(f"Course '{course_title}' not found in year {self.year_path}.")
+            raise CourseUnavailable(f"No such course found: {name}")
-
+        suffix = course_link["href"].replace(f"course/{self.start}-{self.year}", "")
-    from bs4 import BeautifulSoup
+        course_url = self.url + suffix
-
+        return Course(course_url, name, self._session, self)
-    def get_course_by_tag(self, course_tag: str) -> Course:
-        """
-        Gets a course by its tag (course identifier).
-        Constructs the course URL using the year and course tag.
-        """
-        course_path = f"/{self.year_path}/{course_tag}"
-        course_url = f"{self.base_url}/course{course_path}"
-
-        response = self.session.get(course_url)
-        if response.status_code != 200:
-            raise ConnectionError(f"Failed to retrieve course with tag '{course_tag}' for year {self.year_path}. Tried {course_url}")
-
-        soup = BeautifulSoup(response.text, "lxml")
-
-        title_element = soup.find("h1")
-        if not title_element:
-            title_elements = soup.find_all("a", class_="fill accent large")
-            if title_elements:
-                title_element = title_elements[-1]
-
-        if title_element:
-            course_title = title_element.get_text(strip=True)
-        else:
-            raise ValueError(f"Could not retrieve course title for tag '{course_tag}' in year {self.year_path}.")
-
-        return Course(self.session, course_path, course_title, self)
-
-    def __str__(self):
-        return f"Year({self.year_path})"