Lokasi ngalangkungan proxy:   [ UP ]  
[Ngawartoskeun bug]   [Panyetelan cookie]                
Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: datajoint/datajoint-python
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: datajoint/datajoint-python
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: feat/1424-self-upstream
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 8 commits
  • 9 files changed
  • 2 contributors

Commits on Jun 10, 2026

  1. fix(#1429): cascade through FK chain for part_integrity="cascade"

    Diagram.cascade(part_integrity="cascade") used to derive each Part's
    Master restriction by joining `master_ft.proj() & child_ft.proj()` on
    shared attribute names. This failed when the Part referenced its Master
    indirectly — through another Part with renamed FK columns (`.proj()` in
    the definition) or via a Part-of-Part chain that does not directly
    inherit Master's PK names. The intermediate Parts' restrictions were
    also skipped.
    
    Replace the proj-join shortcut with an upward walk of the actual FK
    graph from the Part to its Master, applying symmetric (upward)
    counterparts of the existing propagation rules at each edge.
    
    Key changes in src/datajoint/diagram.py:
    
    - New Diagram._apply_propagation_rule_upward — mirror of the existing
      forward propagation method. Same three rules (shared-PK copy, aliased
      reverse-rename, non-aliased projection) applied in the reverse
      direction (child → parent).
    
    - New Diagram._propagate_part_to_master — walks nx.shortest_path
      (Master → Part) and applies the upward rules along each real edge,
      transparently skipping the integer-named alias nodes that the graph
      inserts for aliased FKs. Restricts intermediate Parts too (the chain
      case from #1429 Case 2). Materializes the Master's restriction via
      to_arrays() so the subsequent forward cascade back down to Master's
      other Parts produces literal `WHERE ... IN (values)` clauses rather
      than self-referential subqueries (avoiding MySQL error 1093).
    
    - New Diagram._find_real_edge_props — looks up edge props for parent →
      child via the direct edge OR through an alias node.
    
    - _propagate_restrictions: seed-is-Part case. When the cascade starts
      at a Part (e.g. `Master.PartB.delete(part_integrity="cascade")`),
      the main loop's part_integrity block — nested inside the out_edges
      iteration — cannot fire because a leaf Part has no out-edges. Trigger
      the upward propagation explicitly for the seed before the main loop.
    
    - Diagram.cascade: expand nodes_to_show to include any node that the
      part_integrity propagation pulled in (the master and its descendants),
      so counts() and __iter__ report the full cascade subgraph.
    
    Tests in tests/integration/test_cascade_delete.py — three new mysql
    tests covering both #1429 cases plus an end-to-end delete. Full
    regression: 8 + 15 + 33 mysql tests pass.
    
    Slated for DataJoint 2.3.
    dimitri-yatsenko committed Jun 10, 2026
    Configuration menu
    Copy the full SHA
    bbbbadf View commit details
    Browse the repository at this point in the history
  2. fix(#1429): address MilagrosMarin review on #1468

    - Drop unused `propagated_edges` parameter from `_propagate_part_to_master`
      and its call sites. The parameter was vestigial after the design
      switched from edge-blocking to materialization at the master.
    
    - Document two limitations in the docstring:
      - Single FK path: nx.shortest_path returns one path; non-shortest
        paths are not applied.
      - Memory cost of materialization: to_arrays() pulls matching master
        PKs into Python memory.
    
    - Add test_cascade_three_level_part_chain covering PartC → PartB →
      PartA → Master. Confirms intermediate Parts are restricted at every
      hop, not just the first.
    
    All 36 mysql tests in cascade_delete + cascading_delete + dependencies
    + semantic_matching pass.
    dimitri-yatsenko committed Jun 10, 2026
    Configuration menu
    Copy the full SHA
    9094a64 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    df8d73d View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2026

  1. feat(#1423): Diagram.trace() for upstream restriction propagation

    Implements T2.2.a of the provenance trinity (datajoint-docs#183). Upstream
    mirror of Diagram.cascade(): walks the FK graph from a restricted seed to
    every ancestor with OR convergence — an ancestor entity is included if
    reachable through any FK path from the seed.
    
    Reuses the upward propagation primitives added by #1468
    (_apply_propagation_rule_upward / _find_real_edge_props) applied here in
    a generalized form (any child → any parent, not just Part → Master).
    
    Branch note: stacked on fix/1429-cascade-part-part-renamed-fk (#1468)
    for the upward primitives. Will rebase onto master after #1468 lands.
    
    What's added:
    
    - src/datajoint/diagram.py:
      - New @classmethod Diagram.trace(table_expr) — mirror of cascade(),
        walks ancestors instead of descendants, trims to ancestor subgraph.
      - New _propagate_restrictions_upstream(start_node) — multi-pass walk
        over in_edges, applies the upward rules at each real edge.
        Alias-node transparent.
      - New __getitem__(key) — supports both Table subclass/instance
        (returns pre-restricted QueryExpression) and string (returns
        pre-restricted FreeTable). Raises DataJointError for tables outside
        the trace's subgraph.
      - Bugfix in _apply_propagation_rule_upward Backward Rule 3: previous
        code projected child to its OWN PK (child_ft.proj()) which excluded
        non-primary FK columns. Now projects to the FK columns via
        proj(*attr_map.keys()), correctly carrying them into the parent
        restriction for non-primary-FK cases. Caught by
        test_trace_or_convergence_two_paths.
    
    - src/datajoint/dependencies.py:
      - New load_all_upstream() — symmetric to load_all_downstream.
        Iteratively discovers upstream schemas reachable via reverse FK
        edges, expanding the graph until convergence.
    
    - src/datajoint/adapters/{base,mysql,postgres}.py:
      - New find_upstream_schemas_sql(schemas_list) on each adapter,
        symmetric to find_downstream_schemas_sql.
    
    - tests/integration/test_trace.py (new, 8 tests covering single-hop,
      multi-hop, renamed FK, OR convergence across two paths, non-ancestor
      rejection, string indexing → FreeTable, counts(), leaf-table seed).
    
    All 8 trace tests pass on MySQL. Regression: test_cascade_delete +
    test_cascading_delete + test_dependencies + test_semantic_matching
    — 36 tests pass, no regressions from the Rule 3 fix.
    
    Slated for DataJoint 2.3.
    dimitri-yatsenko committed Jun 23, 2026
    Configuration menu
    Copy the full SHA
    8d9d242 View commit details
    Browse the repository at this point in the history
  2. fix(#1423): gate Diagram.__getitem__ on _mode == "trace"

    The trace-mode __getitem__ I added shadowed networkx.DiGraph's standard
    adjacency-dict lookup for ALL Diagrams, not just trace results. ERD
    tests (and any other code that does diagram[node_name] for adjacency)
    were getting DataJointError("not in this trace's subgraph") instead of
    the adjacency dict.
    
    Fix: short-circuit non-trace diagrams (no _mode attribute or _mode != "trace")
    to super().__getitem__(key) before any trace-specific logic runs.
    
    Tests:
    - 5 previously-failing erd tests now pass (test_erd, test_diagram_algebra,
      test_repr_svg, test_make_image, test_part_table_parsing).
    - 8/8 trace tests still pass.
    dimitri-yatsenko committed Jun 23, 2026
    Configuration menu
    Copy the full SHA
    03dd0a7 View commit details
    Browse the repository at this point in the history
  3. feat(#1424): self.upstream property for pre-restricted ancestor access

    Implements T2.2.b of the provenance trinity. Inside make(), self.upstream
    exposes a pre-constructed Diagram.trace(self & key) so users can read
    declared ancestors with provenance-safe, ergonomic syntax.
    
    Branch stacked on feat/1423-diagram-trace (#1471) for Diagram.trace().
    
    What's added:
    
    - src/datajoint/autopopulate.py:
      - AutoPopulate._upstream class attribute (default None) — instance
        storage for the per-make() trace.
      - AutoPopulate.upstream property — returns the trace if set, raises
        DataJointError with a clear "only available inside make()" message
        otherwise. The error includes the fallback pattern
        (dj.Diagram.trace(self & key)) so the user knows the escape hatch.
      - In _populate_one, set self._upstream = Diagram.trace(self & dict(key))
        immediately before the make() invocation block. Construction is
        lazy at this layer (graph copy only); the SQL fetch fires when the
        user accesses self.upstream[T].fetch(...).
      - The existing `finally` block (line 716) that resets _allow_insert
        now also resets _upstream to None, so subsequent attribute access
        raises a clear error rather than silently returning a stale trace
        from the previous make() call.
    
    What's not changed:
    
    - make() signature: unchanged — key remains a dict, make_kwargs work
      as before. self.upstream is a new attribute on self, not a new
      parameter.
    - Tripartite make pattern: self._upstream is set once before all three
      make() invocations, so all three phases see the same upstream view.
    
    Tests in tests/integration/test_autopopulate.py (5 new):
    
    - test_upstream_provides_pre_restricted_ancestor — basic case: make()
      reads self.upstream[Subject].fetch1("name") and the value is correctly
      pre-restricted to the current key.
    - test_upstream_rejects_non_ancestor — self.upstream[Unrelated] raises
      DataJointError ("not in this trace"). Inherited from Diagram.__getitem__.
    - test_upstream_unset_outside_make — accessing the property outside of
      make() raises with the helpful "only available inside make()" message.
    - test_upstream_cleared_after_make — after populate() completes, accessing
      the property on a fresh instance still raises (verifies the finally
      cleanup; no stale state).
    - test_upstream_seen_across_tripartite_make — both make_fetch / make_compute
      / make_insert see the same self.upstream value.
    
    Full regression: 17/17 autopopulate tests pass on MySQL.
    
    Slated for DataJoint 2.3. Blocked on #1471 (Diagram.trace) merging;
    T2.2.c (strict_provenance) is stacked on this PR.
    dimitri-yatsenko committed Jun 23, 2026
    Configuration menu
    Copy the full SHA
    7e4130f View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2026

  1. Configuration menu
    Copy the full SHA
    52ff2ea View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b61abf2 View commit details
    Browse the repository at this point in the history
Loading