Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[obsolete?] Fractal with bconds nref1 via references #487

Open
wants to merge 87 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
c5a37c0
sgs_fra solver declaration
pdziekan Feb 28, 2022
f915875
wip on fractal refinement
pdziekan Mar 1, 2022
c1b497d
working allocation of refined arrays
pdziekan Mar 2, 2022
487b356
refined grid size dependant on the number of fractal reconstructions
pdziekan Mar 2, 2022
1ad25f3
fix refine_grid_size()
pdziekan Mar 2, 2022
f3885aa
mem_factory function
pdziekan Mar 2, 2022
47348d7
consitional choice of shmem / shmem_refined
pdziekan Mar 2, 2022
a6e8b43
consitional choice of shmem / shmem_refined - upgraded
pdziekan Mar 3, 2022
be93f45
working shmem / shmem_recon stuff? (openmp and 3d only for now)
pdziekan Mar 3, 2022
963be45
solver_family protected again + solver doesnt inherit from family tag
pdziekan Mar 3, 2022
d361919
use mem_factory in all concurr types
pdziekan Mar 3, 2022
2d956d1
second ctor for shmem_refined in mem_t in each concurr type
pdziekan Mar 3, 2022
7d54514
alloc tmp with custom grid size in 1d and 2d
pdziekan Mar 3, 2022
3f81179
hdf5: store refined data
pdziekan Mar 7, 2022
71f4d70
xmf: record refined aux
pdziekan Mar 7, 2022
0fe0730
hdf5: store refined X,Y,Z TODO: make it conditional
pdziekan Mar 7, 2022
caaffb4
xmf writer: add reference writing
pdziekan Mar 7, 2022
1beeef1
xmf: write refined data using separate xmf_writer
pdziekan Mar 7, 2022
cb7d541
Merge branch 'master' of github.com:igfuw/libmpdataxx into fractal
pdziekan Mar 15, 2022
54e392b
fix grid size refinement strategy
pdziekan Mar 16, 2022
fb08f52
wip on refinement: fixed positions in output, added refined ijk
pdziekan Mar 16, 2022
e9984b5
refined fields without halos
pdziekan Mar 17, 2022
3f540a9
refined grid: halo of n_ref/2
pdziekan Mar 17, 2022
87df456
Revert "refined grid: halo of n_ref/2"
pdziekan Mar 17, 2022
aba74b3
wip on refined interpolation
pdziekan Mar 17, 2022
6aab97c
wip on interpolation
pdziekan Mar 18, 2022
e622df1
interpolation: hardocode two iterations, still no interpolation at sh…
pdziekan Mar 18, 2022
669194a
wip on interpolation at edges
pdziekan Mar 21, 2022
fe9d8f8
shmem_ref: require grid size that makes fractal reconstruction easier
pdziekan Mar 31, 2022
275789c
refinement: wip
pdziekan Mar 31, 2022
e5213f1
refinement: fix interpolation for distmem!
pdziekan Apr 1, 2022
5225da2
working interpolation of scalars
pdziekan Apr 19, 2022
c7eb23a
pbl: refine u too
pdziekan Apr 19, 2022
b0fa3f3
fra rec wip
pdziekan Apr 20, 2022
94af7cb
fra rec in 3 steps:
pdziekan Apr 20, 2022
2701722
fra rec wip
pdziekan Apr 20, 2022
93294f2
fra rec wip
pdziekan Apr 21, 2022
9866e7c
fra rec wip
pdziekan Apr 25, 2022
ab01ef0
new ref ranges working interpolation
pdziekan Apr 25, 2022
c620de4
cleanup
pdziekan Apr 25, 2022
eeaba3f
cleanup + wip on reconstruction
pdziekan Apr 25, 2022
34337eb
working reconstruction? at least for 1 iteration
pdziekan Apr 26, 2022
6c06aed
working recon, at least 1 iter
pdziekan Apr 26, 2022
42c26ac
working recon
pdziekan Apr 27, 2022
72ead36
reconstruction and interpolation: move similar code to functions
pdziekan Apr 27, 2022
12766ed
random stretching parameter
pdziekan Apr 28, 2022
549a592
move fractal reconstruction formulas to formulae/
pdziekan Apr 29, 2022
0fd42ec
Revert "move fractal reconstruction formulas to formulae/"
pdziekan Apr 29, 2022
59b3bab
Revert "Revert "move fractal reconstruction formulas to formulae/""
pdziekan Apr 29, 2022
a788a4d
pbl smg: set fra iter
pdziekan Apr 29, 2022
34ad9a5
pbl smg: fra iter 4
pdziekan Apr 29, 2022
be4a6a3
alloc refined arrays only for required advectees
pdziekan May 4, 2022
6e5ba14
grid refinement: refined boundary in the middle between mpi domains
pdziekan May 9, 2022
4275cea
cleanup
pdziekan May 9, 2022
079a75e
comments
pdziekan Jun 6, 2022
36452f9
include <random> in fractal reconstruction
pdziekan Sep 27, 2022
10a335d
stop on first compilation error in Debug mode
pdziekan Sep 27, 2022
60cd21f
sgs fra make typdefs public
pdziekan Sep 27, 2022
e5e4879
hdf5 output: handle sgs_fra family tag
pdziekan Sep 28, 2022
2c532ec
domain decmposition functions static
pdziekan Sep 28, 2022
eee8135
add hook_ante_record_all
pdziekan Sep 28, 2022
5015952
hook_pre_record_all -> hook_ante_record_all
pdziekan Sep 28, 2022
0194299
barrier in output common
pdziekan Sep 28, 2022
bbe52f9
interpolate ref: dont calc stretchin params
pdziekan Sep 30, 2022
c9b3b62
record prof refined
pdziekan Oct 3, 2022
be08975
comment
pdziekan Oct 5, 2022
815377c
formulae for averaging from refined to regular grid
pdziekan Oct 5, 2022
a7e4a9f
formulae for averaging from refined to regular grid: working but with…
pdziekan Oct 5, 2022
9217fd4
averaging from refined to regular: fixes for edge cells
pdziekan Oct 18, 2022
04ce94d
hdf output: record_aux_refined
pdziekan Oct 19, 2022
bf7e3ca
refined arrays: add halo size 1
pdziekan Oct 19, 2022
d97fc3e
additional set of bconds for refined arrays
pdziekan Oct 21, 2022
45e0dff
TEMP: add refinement to pbl
pdziekan Oct 21, 2022
f5adbd8
concurr_reg for separate init()
pdziekan Oct 21, 2022
bbeb89b
refined courant spatial interpolation
pdziekan Oct 24, 2022
199d9c9
reconstruction: barrier after stretching parameters are created
pdziekan Oct 25, 2022
7194a95
fix remote refined bconds
pdziekan Oct 26, 2022
3577129
mpi grid refinemet wip
pdziekan Oct 27, 2022
900c07c
xchng_ref
pdziekan Oct 28, 2022
e4dc451
delete debug output
pdziekan Oct 28, 2022
804af0b
for doubles use mt19937_64
pdziekan Nov 24, 2022
30114ab
do not generate stretching parameters at each reconsteuction, make us…
pdziekan Nov 24, 2022
e61c945
do not generate stretching parameters at each reconsteuction, make us…
pdziekan Nov 24, 2022
ddde107
track conc_reg
pdziekan Nov 30, 2022
0d5c838
Merge branch 'fractal_with_bconds' of github.com:pdziekan/libmpdataxx…
pdziekan Nov 30, 2022
ab489d8
fixes for non-mpi compilation
Nov 30, 2022
cd34492
WIP: for n_ref==1 make ref variables reference normal variables
pdziekan Jan 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions libmpdata++-config.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ set(libmpdataxx_INCLUDE_DIRS "${CMAKE_CURRENT_LIST_DIR}/../../include/")

############################################################################################
# debug mode compiler flags
set(libmpdataxx_CXX_FLAGS_DEBUG "${libmpdataxx_CXX_FLAGS_DEBUG} -std=c++14 -DBZ_DEBUG -g -Wno-enum-compare") #TODO: -Og if compiler supports it?
set(libmpdataxx_CXX_FLAGS_DEBUG "${libmpdataxx_CXX_FLAGS_DEBUG} -std=c++17 -DBZ_DEBUG -g -Wno-enum-compare -Wfatal-errors") #TODO: -Og if compiler supports it?


############################################################################################
Expand All @@ -42,7 +42,7 @@ if(
CMAKE_CXX_COMPILER_ID STREQUAL "Clang" OR
CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang"
)
set(libmpdataxx_CXX_FLAGS_RELEASE "${libmpdataxx_CXX_FLAGS_RELEASE} -std=c++14 -DNDEBUG -Ofast -march=native -Wno-enum-compare")
set(libmpdataxx_CXX_FLAGS_RELEASE "${libmpdataxx_CXX_FLAGS_RELEASE} -std=c++17 -DNDEBUG -Ofast -march=native -Wno-enum-compare")

# preventing Kahan summation from being optimised out
if (
Expand All @@ -58,7 +58,7 @@ if(
CMAKE_CXX_COMPILER_ID STREQUAL "Intel"
)
# flags taken from -fast but without -static
set(libmpdataxx_CXX_FLAGS_RELEASE "${libmpdataxx_CXX_FLAGS_RELEASE} -std=gnu++14 -DNDEBUG -xHOST -O3 -ipo -no-prec-div -fp-model fast=2")
set(libmpdataxx_CXX_FLAGS_RELEASE "${libmpdataxx_CXX_FLAGS_RELEASE} -std=gnu++17 -DNDEBUG -xHOST -O3 -ipo -no-prec-div -fp-model fast=2")
endif()


Expand Down
5 changes: 2 additions & 3 deletions libmpdata++/bcond/detail/remote_3d_common.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,8 @@ namespace libmpdataxx
// based on the difference between idx to be sent by this thread and idx of this process
idx_t extend_idx(idx_t idx)
{
//std::cerr << "extend idx start idx(1): " << idx.lbound(1) << ", " << idx.ubound(1) << std::endl;
idx.lbound(1) = 0 + idx.lbound(1) - thread_j.first(); // does it have to start at 0?
idx.ubound(1) = (grid_size_y - 1) + idx.ubound(1) - thread_j.last(); // does it have to end at grid_size_y - 1?
//std::cerr << "extend idx end idx(1): " << idx.lbound(1) << ", " << idx.ubound(1) << std::endl;
return idx;
}

Expand Down Expand Up @@ -68,13 +66,14 @@ namespace libmpdataxx

// ctor
remote_3d_common(
boost::mpi::communicator &mpic,
const rng_t &i,
const std::array<int, 3> &distmem_grid_size,
const rng_t _thread_j,
const int thread_rank,
const int thread_size
) :
parent_t(i, distmem_grid_size, true, thread_rank, thread_size), // true indicating that this is a bcond done with a single thread
parent_t(mpic, i, distmem_grid_size, true, thread_rank, thread_size), // true indicating that this is a bcond done with a single thread
thread_j(_thread_j),
grid_size_y(distmem_grid_size[1])
{
Expand Down
25 changes: 4 additions & 21 deletions libmpdata++/bcond/detail/remote_common.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -78,12 +78,6 @@ namespace libmpdataxx
const int
msg_send = dir == left ? left : rght;

// std::cerr << "send_hlpr idx dir " << dir << " : "
// << " (" << idx_send.lbound(0) << ", " << idx_send.ubound(0) << ")"
// << " (" << idx_send.lbound(1) << ", " << idx_send.ubound(1) << ")"
// << " (" << idx_send.lbound(2) << ", " << idx_send.ubound(2) << ")"
// << std::endl;

// arr_send references part of the send buffer that will be used
arr_t arr_send(buf_send, a(idx_send).shape(), blitz::neverDeleteData);
// copying data to be sent
Expand Down Expand Up @@ -118,13 +112,6 @@ namespace libmpdataxx
const int
msg_recv = dir == left ? rght : left;

// std::cerr << "recv_hlpr idx dir " << dir << " : "
// << " (" << idx_recv.lbound(0) << ", " << idx_recv.ubound(0) << ")"
// << " (" << idx_recv.lbound(1) << ", " << idx_recv.ubound(1) << ")"
// << " (" << idx_recv.lbound(2) << ", " << idx_recv.ubound(2) << ")"
// << std::endl;


// launching async data transfer
if(a(idx_recv).size()!=0) // TODO: test directly size of idx_recv
{
Expand Down Expand Up @@ -218,24 +205,20 @@ namespace libmpdataxx

// ctor
remote_common(
boost::mpi::communicator &mpic,
const rng_t &i,
const std::array<int, n_dims> &distmem_grid_size,
bool single_threaded = false,
const int thread_rank = -1,
const int thread_size = -1
const int thread_size = -1
) :
parent_t(i, distmem_grid_size, single_threaded, thread_rank, thread_size)
//,mpicom(mpic, boost::mpi::comm_take_ownership)
,mpicom(mpic, boost::mpi::comm_attach)
{
#if defined(USE_MPI)

const int slice_size = n_dims==1 ? 1 : (n_dims==2? distmem_grid_size[1]+6 : (distmem_grid_size[1]+6) * (distmem_grid_size[2]+6) ); // 3 is the max halo size (?), so 6 on both sides
//std::cerr << "remote_common ctor, "
// << " distmem_grid_size[0]: " << distmem_grid_size[0]
// << " distmem_grid_size[1]: " << distmem_grid_size[1]
// << " distmem_grid_size[2]: " << distmem_grid_size[2]
// << " slice_size: " << slice_size
// << " halo: " << halo
// << std::endl;
// allocate enough memory in buffers to store largest halos to be sent
buf_send = (real_t *) malloc(halo * slice_size * sizeof(real_t));
buf_recv = (real_t *) malloc(halo * slice_size * sizeof(real_t));
Expand Down
1 change: 1 addition & 0 deletions libmpdata++/blitz.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@
namespace libmpdataxx
{
template <int n_dims> using idx_t = blitz::RectDomain<n_dims>;
template <int n_dims> using idxs_t = blitz::StridedDomain<n_dims>;
using rng_t = blitz::Range;

// non-int ix_t means either rng_t or idx_t
Expand Down
8 changes: 3 additions & 5 deletions libmpdata++/concurr/boost_thread.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,8 @@ namespace libmpdataxx


// ctor
mem_t(const std::array<int, solver_t::n_dims> &grid_size) :
b(size(grid_size[0])),
parent_t::mem_t(grid_size, size(grid_size[0]))
{};
mem_t(const std::array<int, solver_t::n_dims> &grid_size, const int n_ref) : b(size(grid_size[0])), parent_t::mem_t(grid_size, size(grid_size[0]), n_ref) {};
mem_t(const std::array<int, solver_t::n_dims> &grid_size) : b(size(grid_size[0])), parent_t::mem_t(grid_size, size(grid_size[0])) {};

void barrier()
{
Expand All @@ -81,7 +79,7 @@ namespace libmpdataxx

// ctor
boost_thread(const typename solver_t::rt_params_t &p) :
parent_t(p, new mem_t(p.grid_size), mem_t::size(p.grid_size[solver_t::n_dims < 3 ? 0 : 1])) // note 3D domain decomposition in y direction
parent_t(p, detail::mem_factory<mem_t, solver_t>(p), mem_t::size(p.grid_size[solver_t::n_dims < 3 ? 0 : 1])) // note 3D domain decomposition in y direction
{}

};
Expand Down
8 changes: 3 additions & 5 deletions libmpdata++/concurr/cxx11_thread.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -92,10 +92,8 @@ namespace libmpdataxx
}

// ctor
mem_t(const std::array<int, solver_t::n_dims> &grid_size) :
b(size(grid_size[0])),
parent_t::mem_t(grid_size, size(grid_size[0]))
{};
mem_t(const std::array<int, solver_t::n_dims> &grid_size, const int n_ref) : b(size(grid_size[0])), parent_t::mem_t(grid_size, size(grid_size[0]), n_ref) {};
mem_t(const std::array<int, solver_t::n_dims> &grid_size) : b(size(grid_size[0])), parent_t::mem_t(grid_size, size(grid_size[0])) {};

void barrier()
{
Expand All @@ -119,7 +117,7 @@ namespace libmpdataxx

// ctor
cxx11_thread(const typename solver_t::rt_params_t &p) :
parent_t(p, new mem_t(p.grid_size), mem_t::size(p.grid_size[solver_t::n_dims < 3 ? 0 : 1])) // note 3D domain decomposition in y direction
parent_t(p, detail::mem_factory<mem_t, solver_t>(p), mem_t::size(p.grid_size[solver_t::n_dims < 3 ? 0 : 1])) // note 3D domain decomposition in y direction
{}

};
Expand Down
Loading